2023-10-27 23:58:10,930 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-10-27 23:58:11,102 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c 2023-10-27 23:58:11,114 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-10-27 23:58:11,114 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/cluster_1ba21e36-f5ca-27f1-0277-ca251e3b747a, deleteOnExit=true 2023-10-27 23:58:11,114 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-10-27 23:58:11,115 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/test.cache.data in system properties and HBase conf 2023-10-27 23:58:11,115 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/hadoop.tmp.dir in system properties and HBase conf 2023-10-27 23:58:11,116 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/hadoop.log.dir in system properties and HBase conf 2023-10-27 23:58:11,116 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/mapreduce.cluster.local.dir in system properties and HBase conf 2023-10-27 23:58:11,117 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-10-27 23:58:11,117 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-10-27 23:58:11,216 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-10-27 23:58:11,521 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-10-27 23:58:11,524 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-10-27 23:58:11,525 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-10-27 23:58:11,525 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-10-27 23:58:11,525 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-10-27 23:58:11,526 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-10-27 23:58:11,526 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-10-27 23:58:11,526 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-10-27 23:58:11,527 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/dfs.journalnode.edits.dir in system properties and HBase conf 2023-10-27 23:58:11,527 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-10-27 23:58:11,527 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/nfs.dump.dir in system properties and HBase conf 2023-10-27 23:58:11,527 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/java.io.tmpdir in system properties and HBase conf 2023-10-27 23:58:11,528 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/dfs.journalnode.edits.dir in system properties and HBase conf 2023-10-27 23:58:11,528 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-10-27 23:58:11,528 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-10-27 23:58:11,942 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-10-27 23:58:11,946 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-10-27 23:58:12,376 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-10-27 23:58:12,507 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-10-27 23:58:12,520 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-10-27 23:58:12,549 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-10-27 23:58:12,575 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/java.io.tmpdir/Jetty_localhost_35019_hdfs____w8jfk/webapp 2023-10-27 23:58:12,676 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35019 2023-10-27 23:58:12,684 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-10-27 23:58:12,684 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-10-27 23:58:13,252 WARN [Listener at localhost/44261] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-10-27 23:58:13,303 WARN [Listener at localhost/44261] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-10-27 23:58:13,318 WARN [Listener at localhost/44261] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-10-27 23:58:13,323 INFO [Listener at localhost/44261] log.Slf4jLog(67): jetty-6.1.26 2023-10-27 23:58:13,327 INFO [Listener at localhost/44261] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/java.io.tmpdir/Jetty_localhost_46085_datanode____de3pw2/webapp 2023-10-27 23:58:13,402 INFO [Listener at localhost/44261] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46085 2023-10-27 23:58:13,642 WARN [Listener at localhost/40295] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-10-27 23:58:14,362 DEBUG [Listener at localhost/40295] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c 2023-10-27 23:58:14,369 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa631ce321de802e3: Processing first storage report for DS-a3de6215-dc4a-4069-a678-19792d7ea06c from datanode 0a9911f5-af3f-4cd3-a291-4cd50df9aa82 2023-10-27 23:58:14,370 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa631ce321de802e3: from storage DS-a3de6215-dc4a-4069-a678-19792d7ea06c node DatanodeRegistration(127.0.0.1:42663, datanodeUuid=0a9911f5-af3f-4cd3-a291-4cd50df9aa82, infoPort=35571, infoSecurePort=0, ipcPort=40295, storageInfo=lv=-57;cid=testClusterID;nsid=1073982801;c=1698451092001), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-10-27 23:58:14,370 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa631ce321de802e3: Processing first storage report for DS-51046459-6680-437d-b3d6-021d8fa29ba4 from datanode 0a9911f5-af3f-4cd3-a291-4cd50df9aa82 2023-10-27 23:58:14,371 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa631ce321de802e3: from storage DS-51046459-6680-437d-b3d6-021d8fa29ba4 node DatanodeRegistration(127.0.0.1:42663, datanodeUuid=0a9911f5-af3f-4cd3-a291-4cd50df9aa82, infoPort=35571, infoSecurePort=0, ipcPort=40295, storageInfo=lv=-57;cid=testClusterID;nsid=1073982801;c=1698451092001), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-10-27 23:58:14,413 INFO [Listener at localhost/40295] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/cluster_1ba21e36-f5ca-27f1-0277-ca251e3b747a/zookeeper_0, clientPort=57369, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/cluster_1ba21e36-f5ca-27f1-0277-ca251e3b747a/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/cluster_1ba21e36-f5ca-27f1-0277-ca251e3b747a/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-10-27 23:58:14,426 INFO [Listener at localhost/40295] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=57369 2023-10-27 23:58:14,437 INFO [Listener at localhost/40295] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-10-27 23:58:14,439 INFO [Listener at localhost/40295] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-10-27 23:58:15,072 INFO [Listener at localhost/40295] util.FSUtils(471): Created version file at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234 with version=8 2023-10-27 23:58:15,072 INFO [Listener at localhost/40295] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/hbase-staging 2023-10-27 23:58:15,340 INFO [Listener at localhost/40295] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-10-27 23:58:15,700 INFO [Listener at localhost/40295] client.ConnectionUtils(127): master/jenkins-hbase16:0 server-side Connection retries=3 2023-10-27 23:58:15,725 INFO [Listener at localhost/40295] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-10-27 23:58:15,725 INFO [Listener at localhost/40295] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-10-27 23:58:15,725 INFO [Listener at localhost/40295] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-10-27 23:58:15,725 INFO [Listener at localhost/40295] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-10-27 23:58:15,726 INFO [Listener at localhost/40295] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-10-27 23:58:15,840 INFO [Listener at localhost/40295] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-10-27 23:58:15,896 DEBUG [Listener at localhost/40295] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-10-27 23:58:15,971 INFO [Listener at localhost/40295] ipc.NettyRpcServer(120): Bind to /188.40.62.62:36413 2023-10-27 23:58:15,981 INFO [Listener at localhost/40295] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-10-27 23:58:15,984 INFO [Listener at localhost/40295] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-10-27 23:58:16,006 INFO [Listener at localhost/40295] zookeeper.RecoverableZooKeeper(93): Process identifier=master:36413 connecting to ZooKeeper ensemble=127.0.0.1:57369 2023-10-27 23:58:16,111 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:364130x0, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-10-27 23:58:16,114 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:36413-0x1018a51ee3f0000 connected 2023-10-27 23:58:16,189 DEBUG [Listener at localhost/40295] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-10-27 23:58:16,190 DEBUG [Listener at localhost/40295] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-10-27 23:58:16,195 DEBUG [Listener at localhost/40295] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-10-27 23:58:16,205 DEBUG [Listener at localhost/40295] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36413 2023-10-27 23:58:16,205 DEBUG [Listener at localhost/40295] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36413 2023-10-27 23:58:16,206 DEBUG [Listener at localhost/40295] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36413 2023-10-27 23:58:16,206 DEBUG [Listener at localhost/40295] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36413 2023-10-27 23:58:16,206 DEBUG [Listener at localhost/40295] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36413 2023-10-27 23:58:16,211 INFO [Listener at localhost/40295] master.HMaster(444): hbase.rootdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234, hbase.cluster.distributed=false 2023-10-27 23:58:16,272 INFO [Listener at localhost/40295] client.ConnectionUtils(127): regionserver/jenkins-hbase16:0 server-side Connection retries=3 2023-10-27 23:58:16,273 INFO [Listener at localhost/40295] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-10-27 23:58:16,273 INFO [Listener at localhost/40295] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-10-27 23:58:16,273 INFO [Listener at localhost/40295] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-10-27 23:58:16,273 INFO [Listener at localhost/40295] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-10-27 23:58:16,273 INFO [Listener at localhost/40295] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-10-27 23:58:16,277 INFO [Listener at localhost/40295] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-10-27 23:58:16,280 INFO [Listener at localhost/40295] ipc.NettyRpcServer(120): Bind to /188.40.62.62:43469 2023-10-27 23:58:16,281 INFO [Listener at localhost/40295] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-10-27 23:58:16,286 DEBUG [Listener at localhost/40295] mob.MobFileCache(121): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-10-27 23:58:16,287 INFO [Listener at localhost/40295] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-10-27 23:58:16,290 INFO [Listener at localhost/40295] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-10-27 23:58:16,292 INFO [Listener at localhost/40295] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:43469 connecting to ZooKeeper ensemble=127.0.0.1:57369 2023-10-27 23:58:16,304 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:434690x0, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-10-27 23:58:16,305 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:43469-0x1018a51ee3f0001 connected 2023-10-27 23:58:16,305 DEBUG [Listener at localhost/40295] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-10-27 23:58:16,307 DEBUG [Listener at localhost/40295] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-10-27 23:58:16,308 DEBUG [Listener at localhost/40295] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-10-27 23:58:16,309 DEBUG [Listener at localhost/40295] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43469 2023-10-27 23:58:16,309 DEBUG [Listener at localhost/40295] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43469 2023-10-27 23:58:16,310 DEBUG [Listener at localhost/40295] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43469 2023-10-27 23:58:16,310 DEBUG [Listener at localhost/40295] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43469 2023-10-27 23:58:16,311 DEBUG [Listener at localhost/40295] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43469 2023-10-27 23:58:16,313 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase16.apache.org,36413,1698451095183 2023-10-27 23:58:16,329 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-10-27 23:58:16,331 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase16.apache.org,36413,1698451095183 2023-10-27 23:58:16,354 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-10-27 23:58:16,354 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-10-27 23:58:16,354 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-10-27 23:58:16,355 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-10-27 23:58:16,356 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-10-27 23:58:16,356 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase16.apache.org,36413,1698451095183 from backup master directory 2023-10-27 23:58:16,362 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase16.apache.org,36413,1698451095183 2023-10-27 23:58:16,362 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-10-27 23:58:16,363 WARN [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-10-27 23:58:16,363 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase16.apache.org,36413,1698451095183 2023-10-27 23:58:16,366 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-10-27 23:58:16,368 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-10-27 23:58:16,448 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/hbase.id with ID: f7496793-44f8-4932-97a3-1b4e34a71f10 2023-10-27 23:58:16,497 INFO [master/jenkins-hbase16:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-10-27 23:58:16,520 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-10-27 23:58:16,562 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x1d87fafe to 127.0.0.1:57369 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-10-27 23:58:16,604 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7cd18d8d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-27 23:58:16,624 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-10-27 23:58:16,626 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-10-27 23:58:16,639 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-10-27 23:58:16,639 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-10-27 23:58:16,641 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-10-27 23:58:16,644 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-10-27 23:58:16,645 INFO [master/jenkins-hbase16:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-10-27 23:58:16,671 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/MasterData/data/master/store-tmp 2023-10-27 23:58:17,102 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:58:17,103 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-10-27 23:58:17,103 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-10-27 23:58:17,103 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-10-27 23:58:17,103 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-10-27 23:58:17,103 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-10-27 23:58:17,103 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-10-27 23:58:17,103 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-10-27 23:58:17,105 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/MasterData/WALs/jenkins-hbase16.apache.org,36413,1698451095183 2023-10-27 23:58:17,126 INFO [master/jenkins-hbase16:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase16.apache.org%2C36413%2C1698451095183, suffix=, logDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/MasterData/WALs/jenkins-hbase16.apache.org,36413,1698451095183, archiveDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/MasterData/oldWALs, maxLogs=10 2023-10-27 23:58:17,176 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42663,DS-a3de6215-dc4a-4069-a678-19792d7ea06c,DISK] 2023-10-27 23:58:17,182 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-10-27 23:58:17,233 INFO [master/jenkins-hbase16:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/MasterData/WALs/jenkins-hbase16.apache.org,36413,1698451095183/jenkins-hbase16.apache.org%2C36413%2C1698451095183.1698451097135 2023-10-27 23:58:17,234 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42663,DS-a3de6215-dc4a-4069-a678-19792d7ea06c,DISK]] 2023-10-27 23:58:17,234 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-10-27 23:58:17,234 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:58:17,237 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-10-27 23:58:17,238 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-10-27 23:58:17,288 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-10-27 23:58:17,295 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-10-27 23:58:17,321 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-10-27 23:58:17,332 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-27 23:58:17,338 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-10-27 23:58:17,340 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-10-27 23:58:17,353 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-10-27 23:58:17,357 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-10-27 23:58:17,358 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61543870, jitterRate=-0.08292487263679504}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-10-27 23:58:17,358 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-10-27 23:58:17,359 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-10-27 23:58:17,377 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-10-27 23:58:17,377 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(561): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-10-27 23:58:17,379 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-10-27 23:58:17,381 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 1 msec 2023-10-27 23:58:17,406 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 24 msec 2023-10-27 23:58:17,406 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-10-27 23:58:17,427 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-10-27 23:58:17,432 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-10-27 23:58:17,453 INFO [master/jenkins-hbase16:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-10-27 23:58:17,456 INFO [master/jenkins-hbase16:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-10-27 23:58:17,458 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-10-27 23:58:17,463 INFO [master/jenkins-hbase16:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-10-27 23:58:17,467 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-10-27 23:58:17,504 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-10-27 23:58:17,506 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-10-27 23:58:17,506 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-10-27 23:58:17,520 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-10-27 23:58:17,529 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-10-27 23:58:17,529 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-10-27 23:58:17,529 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-10-27 23:58:17,529 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase16.apache.org,36413,1698451095183, sessionid=0x1018a51ee3f0000, setting cluster-up flag (Was=false) 2023-10-27 23:58:17,562 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-10-27 23:58:17,587 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-10-27 23:58:17,589 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase16.apache.org,36413,1698451095183 2023-10-27 23:58:17,604 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-10-27 23:58:17,629 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-10-27 23:58:17,630 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase16.apache.org,36413,1698451095183 2023-10-27 23:58:17,633 WARN [master/jenkins-hbase16:0:becomeActiveMaster] snapshot.SnapshotManager(304): Couldn't delete working snapshot directory: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.hbase-snapshot/.tmp 2023-10-27 23:58:17,716 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(951): ClusterId : f7496793-44f8-4932-97a3-1b4e34a71f10 2023-10-27 23:58:17,720 DEBUG [RS:0;jenkins-hbase16:43469] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-10-27 23:58:17,729 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-10-27 23:58:17,730 DEBUG [RS:0;jenkins-hbase16:43469] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-10-27 23:58:17,731 DEBUG [RS:0;jenkins-hbase16:43469] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-10-27 23:58:17,738 DEBUG [RS:0;jenkins-hbase16:43469] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-10-27 23:58:17,740 DEBUG [RS:0;jenkins-hbase16:43469] zookeeper.ReadOnlyZKClient(139): Connect 0x36e87475 to 127.0.0.1:57369 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-10-27 23:58:17,741 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-10-27 23:58:17,741 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-10-27 23:58:17,742 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-10-27 23:58:17,742 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-10-27 23:58:17,742 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase16:0, corePoolSize=10, maxPoolSize=10 2023-10-27 23:58:17,742 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-10-27 23:58:17,742 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=2, maxPoolSize=2 2023-10-27 23:58:17,742 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-10-27 23:58:17,743 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1698451127743 2023-10-27 23:58:17,745 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-10-27 23:58:17,748 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-10-27 23:58:17,748 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-10-27 23:58:17,753 DEBUG [RS:0;jenkins-hbase16:43469] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@382f9cd8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-27 23:58:17,754 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-10-27 23:58:17,754 DEBUG [RS:0;jenkins-hbase16:43469] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3c101f05, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase16.apache.org/188.40.62.62:0 2023-10-27 23:58:17,757 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-10-27 23:58:17,766 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-10-27 23:58:17,766 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-10-27 23:58:17,767 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-10-27 23:58:17,767 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-10-27 23:58:17,768 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-10-27 23:58:17,769 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-10-27 23:58:17,770 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-10-27 23:58:17,771 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-10-27 23:58:17,772 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-10-27 23:58:17,773 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-10-27 23:58:17,774 DEBUG [RS:0;jenkins-hbase16:43469] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase16:43469 2023-10-27 23:58:17,774 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.large.0-1698451097774,5,FailOnTimeoutGroup] 2023-10-27 23:58:17,775 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.small.0-1698451097775,5,FailOnTimeoutGroup] 2023-10-27 23:58:17,775 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-10-27 23:58:17,775 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-10-27 23:58:17,776 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-10-27 23:58:17,777 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-10-27 23:58:17,781 INFO [RS:0;jenkins-hbase16:43469] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-10-27 23:58:17,782 INFO [RS:0;jenkins-hbase16:43469] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-10-27 23:58:17,782 DEBUG [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(1022): About to register with Master. 2023-10-27 23:58:17,784 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase16.apache.org,36413,1698451095183 with isa=jenkins-hbase16.apache.org/188.40.62.62:43469, startcode=1698451096272 2023-10-27 23:58:17,797 DEBUG [RS:0;jenkins-hbase16:43469] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-10-27 23:58:17,835 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:60229, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-10-27 23:58:17,844 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=36413] master.ServerManager(388): Registering regionserver=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:17,857 DEBUG [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234 2023-10-27 23:58:17,858 DEBUG [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:44261 2023-10-27 23:58:17,858 DEBUG [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-10-27 23:58:17,870 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-10-27 23:58:17,871 DEBUG [RS:0;jenkins-hbase16:43469] zookeeper.ZKUtil(162): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:17,871 WARN [RS:0;jenkins-hbase16:43469] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-10-27 23:58:17,872 INFO [RS:0;jenkins-hbase16:43469] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-10-27 23:58:17,872 DEBUG [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(1948): logDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/WALs/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:17,873 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase16.apache.org,43469,1698451096272] 2023-10-27 23:58:17,883 DEBUG [RS:0;jenkins-hbase16:43469] zookeeper.ZKUtil(162): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:17,895 DEBUG [RS:0;jenkins-hbase16:43469] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-10-27 23:58:17,902 INFO [RS:0;jenkins-hbase16:43469] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-10-27 23:58:17,918 INFO [RS:0;jenkins-hbase16:43469] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-10-27 23:58:17,921 INFO [RS:0;jenkins-hbase16:43469] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-10-27 23:58:17,921 INFO [RS:0;jenkins-hbase16:43469] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-10-27 23:58:17,922 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-10-27 23:58:17,928 INFO [RS:0;jenkins-hbase16:43469] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-10-27 23:58:17,928 DEBUG [RS:0;jenkins-hbase16:43469] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-10-27 23:58:17,928 DEBUG [RS:0;jenkins-hbase16:43469] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-10-27 23:58:17,928 DEBUG [RS:0;jenkins-hbase16:43469] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-10-27 23:58:17,928 DEBUG [RS:0;jenkins-hbase16:43469] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-10-27 23:58:17,928 DEBUG [RS:0;jenkins-hbase16:43469] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-10-27 23:58:17,929 DEBUG [RS:0;jenkins-hbase16:43469] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase16:0, corePoolSize=2, maxPoolSize=2 2023-10-27 23:58:17,929 DEBUG [RS:0;jenkins-hbase16:43469] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-10-27 23:58:17,929 DEBUG [RS:0;jenkins-hbase16:43469] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-10-27 23:58:17,929 DEBUG [RS:0;jenkins-hbase16:43469] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-10-27 23:58:17,929 DEBUG [RS:0;jenkins-hbase16:43469] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-10-27 23:58:17,930 INFO [RS:0;jenkins-hbase16:43469] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-10-27 23:58:17,930 INFO [RS:0;jenkins-hbase16:43469] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-10-27 23:58:17,930 INFO [RS:0;jenkins-hbase16:43469] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-10-27 23:58:17,943 INFO [RS:0;jenkins-hbase16:43469] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-10-27 23:58:17,945 INFO [RS:0;jenkins-hbase16:43469] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,43469,1698451096272-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-10-27 23:58:17,958 INFO [RS:0;jenkins-hbase16:43469] regionserver.Replication(203): jenkins-hbase16.apache.org,43469,1698451096272 started 2023-10-27 23:58:17,958 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(1637): Serving as jenkins-hbase16.apache.org,43469,1698451096272, RpcServer on jenkins-hbase16.apache.org/188.40.62.62:43469, sessionid=0x1018a51ee3f0001 2023-10-27 23:58:17,958 DEBUG [RS:0;jenkins-hbase16:43469] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-10-27 23:58:17,959 DEBUG [RS:0;jenkins-hbase16:43469] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:17,959 DEBUG [RS:0;jenkins-hbase16:43469] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase16.apache.org,43469,1698451096272' 2023-10-27 23:58:17,959 DEBUG [RS:0;jenkins-hbase16:43469] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:17,960 DEBUG [RS:0;jenkins-hbase16:43469] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:17,960 DEBUG [RS:0;jenkins-hbase16:43469] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-10-27 23:58:17,960 DEBUG [RS:0;jenkins-hbase16:43469] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-10-27 23:58:17,960 DEBUG [RS:0;jenkins-hbase16:43469] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:17,960 DEBUG [RS:0;jenkins-hbase16:43469] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase16.apache.org,43469,1698451096272' 2023-10-27 23:58:17,960 DEBUG [RS:0;jenkins-hbase16:43469] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-10-27 23:58:17,961 DEBUG [RS:0;jenkins-hbase16:43469] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-10-27 23:58:17,962 DEBUG [RS:0;jenkins-hbase16:43469] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-10-27 23:58:17,962 INFO [RS:0;jenkins-hbase16:43469] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-10-27 23:58:17,962 INFO [RS:0;jenkins-hbase16:43469] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-10-27 23:58:18,078 INFO [RS:0;jenkins-hbase16:43469] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase16.apache.org%2C43469%2C1698451096272, suffix=, logDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/WALs/jenkins-hbase16.apache.org,43469,1698451096272, archiveDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/oldWALs, maxLogs=32 2023-10-27 23:58:18,093 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42663,DS-a3de6215-dc4a-4069-a678-19792d7ea06c,DISK] 2023-10-27 23:58:18,097 INFO [RS:0;jenkins-hbase16:43469] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/WALs/jenkins-hbase16.apache.org,43469,1698451096272/jenkins-hbase16.apache.org%2C43469%2C1698451096272.1698451098080 2023-10-27 23:58:18,097 DEBUG [RS:0;jenkins-hbase16:43469] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42663,DS-a3de6215-dc4a-4069-a678-19792d7ea06c,DISK]] 2023-10-27 23:58:18,188 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-10-27 23:58:18,190 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-10-27 23:58:18,191 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234 2023-10-27 23:58:18,609 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:58:18,613 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-10-27 23:58:18,617 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/1588230740/info 2023-10-27 23:58:18,618 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-10-27 23:58:18,620 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-27 23:58:18,620 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-10-27 23:58:18,624 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/1588230740/rep_barrier 2023-10-27 23:58:18,625 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-10-27 23:58:18,626 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-27 23:58:18,626 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-10-27 23:58:18,629 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/1588230740/table 2023-10-27 23:58:18,629 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-10-27 23:58:18,630 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-27 23:58:18,632 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/1588230740 2023-10-27 23:58:18,633 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/1588230740 2023-10-27 23:58:18,637 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-10-27 23:58:18,639 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-10-27 23:58:18,642 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-10-27 23:58:18,643 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70299828, jitterRate=0.04754906892776489}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-10-27 23:58:18,643 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-10-27 23:58:18,643 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-10-27 23:58:18,643 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-10-27 23:58:18,643 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-10-27 23:58:18,643 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-10-27 23:58:18,643 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-10-27 23:58:18,644 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-10-27 23:58:18,645 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-10-27 23:58:18,650 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-10-27 23:58:18,650 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-10-27 23:58:18,658 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-10-27 23:58:18,667 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-10-27 23:58:18,670 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-10-27 23:58:18,824 DEBUG [jenkins-hbase16:36413] assignment.AssignmentManager(2186): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-10-27 23:58:18,830 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase16.apache.org,43469,1698451096272, state=OPENING 2023-10-27 23:58:18,878 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-10-27 23:58:18,887 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-10-27 23:58:18,888 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-10-27 23:58:18,891 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-27 23:58:19,080 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:19,082 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-10-27 23:58:19,086 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:60202, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-10-27 23:58:19,097 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-10-27 23:58:19,098 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-10-27 23:58:19,102 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase16.apache.org%2C43469%2C1698451096272.meta, suffix=.meta, logDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/WALs/jenkins-hbase16.apache.org,43469,1698451096272, archiveDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/oldWALs, maxLogs=32 2023-10-27 23:58:19,119 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42663,DS-a3de6215-dc4a-4069-a678-19792d7ea06c,DISK] 2023-10-27 23:58:19,124 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/WALs/jenkins-hbase16.apache.org,43469,1698451096272/jenkins-hbase16.apache.org%2C43469%2C1698451096272.meta.1698451099104.meta 2023-10-27 23:58:19,124 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42663,DS-a3de6215-dc4a-4069-a678-19792d7ea06c,DISK]] 2023-10-27 23:58:19,125 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-10-27 23:58:19,126 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-10-27 23:58:19,142 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-10-27 23:58:19,145 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-10-27 23:58:19,151 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-10-27 23:58:19,151 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:58:19,151 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-10-27 23:58:19,151 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-10-27 23:58:19,154 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-10-27 23:58:19,156 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/1588230740/info 2023-10-27 23:58:19,156 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/1588230740/info 2023-10-27 23:58:19,157 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-10-27 23:58:19,158 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-27 23:58:19,158 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-10-27 23:58:19,159 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/1588230740/rep_barrier 2023-10-27 23:58:19,159 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/1588230740/rep_barrier 2023-10-27 23:58:19,160 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-10-27 23:58:19,161 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-27 23:58:19,161 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-10-27 23:58:19,163 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/1588230740/table 2023-10-27 23:58:19,163 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/1588230740/table 2023-10-27 23:58:19,164 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-10-27 23:58:19,164 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-27 23:58:19,166 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/1588230740 2023-10-27 23:58:19,169 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/1588230740 2023-10-27 23:58:19,172 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-10-27 23:58:19,174 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-10-27 23:58:19,175 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66406780, jitterRate=-0.010461866855621338}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-10-27 23:58:19,175 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-10-27 23:58:19,183 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1698451099074 2023-10-27 23:58:19,199 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-10-27 23:58:19,200 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-10-27 23:58:19,200 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase16.apache.org,43469,1698451096272, state=OPEN 2023-10-27 23:58:19,210 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-10-27 23:58:19,211 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-10-27 23:58:19,216 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=3, resume processing ppid=2 2023-10-27 23:58:19,216 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase16.apache.org,43469,1698451096272 in 320 msec 2023-10-27 23:58:19,222 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=2, resume processing ppid=1 2023-10-27 23:58:19,223 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 560 msec 2023-10-27 23:58:19,232 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 1.5560 sec 2023-10-27 23:58:19,232 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1698451099232, completionTime=-1 2023-10-27 23:58:19,232 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.ServerManager(808): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-10-27 23:58:19,233 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(1527): Joining cluster... 2023-10-27 23:58:19,287 DEBUG [hconnection-0x6e7da2c7-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-10-27 23:58:19,291 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:60206, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-10-27 23:58:19,306 INFO [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(1539): Number of RegionServers=1 2023-10-27 23:58:19,307 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1698451159307 2023-10-27 23:58:19,307 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1698451219307 2023-10-27 23:58:19,307 INFO [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(1546): Joined the cluster in 74 msec 2023-10-27 23:58:19,364 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,36413,1698451095183-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-10-27 23:58:19,364 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,36413,1698451095183-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-10-27 23:58:19,364 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,36413,1698451095183-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-10-27 23:58:19,366 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase16:36413, period=300000, unit=MILLISECONDS is enabled. 2023-10-27 23:58:19,366 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-10-27 23:58:19,372 DEBUG [master/jenkins-hbase16:0.Chore.1] janitor.CatalogJanitor(175): 2023-10-27 23:58:19,380 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-10-27 23:58:19,381 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-10-27 23:58:19,390 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-10-27 23:58:19,392 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-10-27 23:58:19,395 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-10-27 23:58:19,415 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/hbase/namespace/fec9ad5a4447830583ac360f9df5b068 2023-10-27 23:58:19,417 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/hbase/namespace/fec9ad5a4447830583ac360f9df5b068 empty. 2023-10-27 23:58:19,418 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/hbase/namespace/fec9ad5a4447830583ac360f9df5b068 2023-10-27 23:58:19,418 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-10-27 23:58:19,847 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-10-27 23:58:19,854 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => fec9ad5a4447830583ac360f9df5b068, NAME => 'hbase:namespace,,1698451099381.fec9ad5a4447830583ac360f9df5b068.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp 2023-10-27 23:58:20,273 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1698451099381.fec9ad5a4447830583ac360f9df5b068.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:58:20,273 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing fec9ad5a4447830583ac360f9df5b068, disabling compactions & flushes 2023-10-27 23:58:20,273 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1698451099381.fec9ad5a4447830583ac360f9df5b068. 2023-10-27 23:58:20,273 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1698451099381.fec9ad5a4447830583ac360f9df5b068. 2023-10-27 23:58:20,274 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1698451099381.fec9ad5a4447830583ac360f9df5b068. after waiting 0 ms 2023-10-27 23:58:20,274 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1698451099381.fec9ad5a4447830583ac360f9df5b068. 2023-10-27 23:58:20,274 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1698451099381.fec9ad5a4447830583ac360f9df5b068. 2023-10-27 23:58:20,274 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for fec9ad5a4447830583ac360f9df5b068: 2023-10-27 23:58:20,284 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-10-27 23:58:20,296 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1698451099381.fec9ad5a4447830583ac360f9df5b068.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1698451100286"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451100286"}]},"ts":"1698451100286"} 2023-10-27 23:58:20,318 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-10-27 23:58:20,319 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-10-27 23:58:20,324 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451100320"}]},"ts":"1698451100320"} 2023-10-27 23:58:20,330 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-10-27 23:58:20,401 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=fec9ad5a4447830583ac360f9df5b068, ASSIGN}] 2023-10-27 23:58:20,406 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=fec9ad5a4447830583ac360f9df5b068, ASSIGN 2023-10-27 23:58:20,407 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=fec9ad5a4447830583ac360f9df5b068, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,43469,1698451096272; forceNewPlan=false, retain=false 2023-10-27 23:58:20,559 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=fec9ad5a4447830583ac360f9df5b068, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:20,560 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1698451099381.fec9ad5a4447830583ac360f9df5b068.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1698451100559"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451100559"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451100559"}]},"ts":"1698451100559"} 2023-10-27 23:58:20,571 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure fec9ad5a4447830583ac360f9df5b068, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-27 23:58:20,726 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:20,739 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1698451099381.fec9ad5a4447830583ac360f9df5b068. 2023-10-27 23:58:20,740 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => fec9ad5a4447830583ac360f9df5b068, NAME => 'hbase:namespace,,1698451099381.fec9ad5a4447830583ac360f9df5b068.', STARTKEY => '', ENDKEY => ''} 2023-10-27 23:58:20,741 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace fec9ad5a4447830583ac360f9df5b068 2023-10-27 23:58:20,741 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1698451099381.fec9ad5a4447830583ac360f9df5b068.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:58:20,741 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for fec9ad5a4447830583ac360f9df5b068 2023-10-27 23:58:20,741 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for fec9ad5a4447830583ac360f9df5b068 2023-10-27 23:58:20,744 INFO [StoreOpener-fec9ad5a4447830583ac360f9df5b068-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region fec9ad5a4447830583ac360f9df5b068 2023-10-27 23:58:20,746 DEBUG [StoreOpener-fec9ad5a4447830583ac360f9df5b068-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/namespace/fec9ad5a4447830583ac360f9df5b068/info 2023-10-27 23:58:20,746 DEBUG [StoreOpener-fec9ad5a4447830583ac360f9df5b068-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/namespace/fec9ad5a4447830583ac360f9df5b068/info 2023-10-27 23:58:20,746 INFO [StoreOpener-fec9ad5a4447830583ac360f9df5b068-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fec9ad5a4447830583ac360f9df5b068 columnFamilyName info 2023-10-27 23:58:20,747 INFO [StoreOpener-fec9ad5a4447830583ac360f9df5b068-1] regionserver.HStore(310): Store=fec9ad5a4447830583ac360f9df5b068/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-27 23:58:20,749 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/namespace/fec9ad5a4447830583ac360f9df5b068 2023-10-27 23:58:20,750 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/namespace/fec9ad5a4447830583ac360f9df5b068 2023-10-27 23:58:20,754 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for fec9ad5a4447830583ac360f9df5b068 2023-10-27 23:58:20,757 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/namespace/fec9ad5a4447830583ac360f9df5b068/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-10-27 23:58:20,758 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened fec9ad5a4447830583ac360f9df5b068; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72412865, jitterRate=0.07903577387332916}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-10-27 23:58:20,758 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for fec9ad5a4447830583ac360f9df5b068: 2023-10-27 23:58:20,759 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1698451099381.fec9ad5a4447830583ac360f9df5b068., pid=6, masterSystemTime=1698451100726 2023-10-27 23:58:20,763 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1698451099381.fec9ad5a4447830583ac360f9df5b068. 2023-10-27 23:58:20,763 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1698451099381.fec9ad5a4447830583ac360f9df5b068. 2023-10-27 23:58:20,765 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=fec9ad5a4447830583ac360f9df5b068, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:20,765 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1698451099381.fec9ad5a4447830583ac360f9df5b068.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1698451100764"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451100764"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451100764"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451100764"}]},"ts":"1698451100764"} 2023-10-27 23:58:20,774 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=6, resume processing ppid=5 2023-10-27 23:58:20,774 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure fec9ad5a4447830583ac360f9df5b068, server=jenkins-hbase16.apache.org,43469,1698451096272 in 199 msec 2023-10-27 23:58:20,778 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=5, resume processing ppid=4 2023-10-27 23:58:20,778 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=fec9ad5a4447830583ac360f9df5b068, ASSIGN in 373 msec 2023-10-27 23:58:20,780 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-10-27 23:58:20,780 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451100780"}]},"ts":"1698451100780"} 2023-10-27 23:58:20,785 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-10-27 23:58:20,822 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-10-27 23:58:20,823 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-10-27 23:58:20,828 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 1.4410 sec 2023-10-27 23:58:20,828 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-10-27 23:58:20,829 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-10-27 23:58:20,864 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-10-27 23:58:20,887 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-10-27 23:58:20,899 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 41 msec 2023-10-27 23:58:20,906 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-10-27 23:58:20,927 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-10-27 23:58:20,941 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 34 msec 2023-10-27 23:58:20,970 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-10-27 23:58:20,987 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-10-27 23:58:20,987 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 4.624sec 2023-10-27 23:58:20,991 INFO [master/jenkins-hbase16:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-10-27 23:58:20,994 INFO [master/jenkins-hbase16:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-10-27 23:58:20,995 INFO [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-10-27 23:58:20,995 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,36413,1698451095183-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-10-27 23:58:20,996 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,36413,1698451095183-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-10-27 23:58:21,006 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-10-27 23:58:21,025 DEBUG [Listener at localhost/40295] zookeeper.ReadOnlyZKClient(139): Connect 0x22679832 to 127.0.0.1:57369 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-10-27 23:58:21,038 DEBUG [Listener at localhost/40295] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@30ed8bb1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-27 23:58:21,054 DEBUG [hconnection-0x22742ed9-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-10-27 23:58:21,064 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:60210, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-10-27 23:58:21,072 INFO [Listener at localhost/40295] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase16.apache.org,36413,1698451095183 2023-10-27 23:58:21,074 DEBUG [Listener at localhost/40295] zookeeper.ReadOnlyZKClient(139): Connect 0x2b262d2b to 127.0.0.1:57369 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-10-27 23:58:21,088 DEBUG [Listener at localhost/40295] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@679a402, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-27 23:58:21,088 INFO [Listener at localhost/40295] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-10-27 23:58:21,108 INFO [Listener at localhost/40295] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=213, OpenFileDescriptor=478, MaxFileDescriptor=60000, SystemLoadAverage=227, ProcessCount=167, AvailableMemoryMB=5165 2023-10-27 23:58:21,118 DEBUG [Listener at localhost/40295] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-10-27 23:58:21,121 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:36216, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-10-27 23:58:21,133 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.HMaster$4(2112): Client=jenkins//188.40.62.62 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-10-27 23:58:21,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure2.ProcedureExecutor(1028): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:21,137 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-10-27 23:58:21,139 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-10-27 23:58:21,141 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(700): Client=jenkins//188.40.62.62 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-10-27 23:58:21,142 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f 2023-10-27 23:58:21,143 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f empty. 2023-10-27 23:58:21,145 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f 2023-10-27 23:58:21,145 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-10-27 23:58:21,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-10-27 23:58:21,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-10-27 23:58:21,461 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-10-27 23:58:21,566 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-10-27 23:58:21,569 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7675): creating {ENCODED => cb61fdd866246978279c7ac1cfc5941f, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp 2023-10-27 23:58:21,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-10-27 23:58:21,983 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:58:21,985 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1604): Closing cb61fdd866246978279c7ac1cfc5941f, disabling compactions & flushes 2023-10-27 23:58:21,985 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:21,986 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:21,986 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. after waiting 0 ms 2023-10-27 23:58:21,986 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:21,987 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:21,987 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1558): Region close journal for cb61fdd866246978279c7ac1cfc5941f: 2023-10-27 23:58:21,994 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-10-27 23:58:21,996 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1698451101996"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451101996"}]},"ts":"1698451101996"} 2023-10-27 23:58:21,999 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-10-27 23:58:22,000 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-10-27 23:58:22,001 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451102000"}]},"ts":"1698451102000"} 2023-10-27 23:58:22,003 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-10-27 23:58:22,046 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=cb61fdd866246978279c7ac1cfc5941f, ASSIGN}] 2023-10-27 23:58:22,049 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=cb61fdd866246978279c7ac1cfc5941f, ASSIGN 2023-10-27 23:58:22,051 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=cb61fdd866246978279c7ac1cfc5941f, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,43469,1698451096272; forceNewPlan=false, retain=false 2023-10-27 23:58:22,203 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=cb61fdd866246978279c7ac1cfc5941f, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:22,204 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1698451102203"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451102203"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451102203"}]},"ts":"1698451102203"} 2023-10-27 23:58:22,208 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure cb61fdd866246978279c7ac1cfc5941f, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-27 23:58:22,269 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-10-27 23:58:22,364 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:22,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:22,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => cb61fdd866246978279c7ac1cfc5941f, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f.', STARTKEY => '', ENDKEY => ''} 2023-10-27 23:58:22,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly cb61fdd866246978279c7ac1cfc5941f 2023-10-27 23:58:22,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:58:22,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for cb61fdd866246978279c7ac1cfc5941f 2023-10-27 23:58:22,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for cb61fdd866246978279c7ac1cfc5941f 2023-10-27 23:58:22,380 INFO [StoreOpener-cb61fdd866246978279c7ac1cfc5941f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region cb61fdd866246978279c7ac1cfc5941f 2023-10-27 23:58:22,383 DEBUG [StoreOpener-cb61fdd866246978279c7ac1cfc5941f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1 2023-10-27 23:58:22,383 DEBUG [StoreOpener-cb61fdd866246978279c7ac1cfc5941f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1 2023-10-27 23:58:22,386 INFO [StoreOpener-cb61fdd866246978279c7ac1cfc5941f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cb61fdd866246978279c7ac1cfc5941f columnFamilyName family1 2023-10-27 23:58:22,390 INFO [StoreOpener-cb61fdd866246978279c7ac1cfc5941f-1] regionserver.HStore(310): Store=cb61fdd866246978279c7ac1cfc5941f/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-27 23:58:22,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f 2023-10-27 23:58:22,394 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f 2023-10-27 23:58:22,398 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for cb61fdd866246978279c7ac1cfc5941f 2023-10-27 23:58:22,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-10-27 23:58:22,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened cb61fdd866246978279c7ac1cfc5941f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71444069, jitterRate=0.06459958851337433}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-10-27 23:58:22,404 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for cb61fdd866246978279c7ac1cfc5941f: 2023-10-27 23:58:22,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f., pid=11, masterSystemTime=1698451102364 2023-10-27 23:58:22,408 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:22,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:22,409 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=cb61fdd866246978279c7ac1cfc5941f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:22,409 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1698451102409"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451102409"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451102409"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451102409"}]},"ts":"1698451102409"} 2023-10-27 23:58:22,416 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=11, resume processing ppid=10 2023-10-27 23:58:22,417 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure cb61fdd866246978279c7ac1cfc5941f, server=jenkins-hbase16.apache.org,43469,1698451096272 in 205 msec 2023-10-27 23:58:22,420 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=10, resume processing ppid=9 2023-10-27 23:58:22,420 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=cb61fdd866246978279c7ac1cfc5941f, ASSIGN in 370 msec 2023-10-27 23:58:22,422 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-10-27 23:58:22,422 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451102422"}]},"ts":"1698451102422"} 2023-10-27 23:58:22,424 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-10-27 23:58:22,429 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-10-27 23:58:22,432 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.2970 sec 2023-10-27 23:58:23,273 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-10-27 23:58:23,274 INFO [Listener at localhost/40295] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-10-27 23:58:23,296 DEBUG [hconnection-0x2bb32ba0-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-10-27 23:58:23,298 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:43928, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-10-27 23:58:23,447 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:23,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:23,453 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-10-27 23:58:23,453 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:23,453 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-10-27 23:58:23,453 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-10-27 23:58:23,454 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:23,454 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:23,512 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:23,512 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:23,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:23,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:23,513 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:23,513 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:23,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:23,514 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:23,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:23,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:23,516 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:23,519 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:23,519 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-10-27 23:58:23,519 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:23,520 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-10-27 23:58:23,521 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:23,522 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:23,522 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:23,522 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. started... 2023-10-27 23:58:23,524 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing cb61fdd866246978279c7ac1cfc5941f 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-10-27 23:58:23,905 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-10-27 23:58:23,912 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-10-27 23:58:23,913 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-10-27 23:58:24,056 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511307a69e454f0c34748a03c93435d80400f to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511307a69e454f0c34748a03c93435d80400f 2023-10-27 23:58:24,475 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/.tmp/family1/ab74b3c95daf4454b233f8a94d42f14e 2023-10-27 23:58:24,490 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/.tmp/family1/ab74b3c95daf4454b233f8a94d42f14e as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/ab74b3c95daf4454b233f8a94d42f14e 2023-10-27 23:58:24,501 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/ab74b3c95daf4454b233f8a94d42f14e, entries=1000, sequenceid=5, filesize=175.8 K 2023-10-27 23:58:24,504 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for cb61fdd866246978279c7ac1cfc5941f in 980ms, sequenceid=5, compaction requested=false 2023-10-27 23:58:24,505 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for cb61fdd866246978279c7ac1cfc5941f: 2023-10-27 23:58:24,505 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:24,505 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:24,506 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:24,506 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:24,506 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-10-27 23:58:24,506 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-10-27 23:58:24,537 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:24,537 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:24,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:24,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:24,538 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,539 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:24,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:24,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:24,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:24,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:24,545 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-10-27 23:58:24,547 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-10-27 23:58:24,547 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5a79467e[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:24,547 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,569 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,570 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-10-27 23:58:24,570 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-10-27 23:58:24,570 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:24,570 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:24,570 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:24,579 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:24,579 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-10-27 23:58:24,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:24,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:24,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:24,580 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:24,581 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-10-27 23:58:24,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:24,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:24,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:24,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:24,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:24,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:58:24,588 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-10-27 23:58:24,588 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-10-27 23:58:24,589 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:24,589 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:24,589 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,589 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:24,595 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,595 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:24,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:24,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,596 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:24,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:24,596 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:24,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:24,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:24,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:24,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,598 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:24,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:24,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:24,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:24,612 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:24,612 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:24,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:24,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:24,612 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:24,612 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,612 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-10-27 23:58:24,612 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:24,612 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,612 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:24,612 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:24,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:24,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:24,613 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,613 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,613 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,615 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:24,615 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:24,679 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-10-27 23:58:24,715 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:24,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:24,731 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:24,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-10-27 23:58:24,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,735 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-10-27 23:58:24,735 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:24,735 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-10-27 23:58:24,735 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-10-27 23:58:24,736 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,736 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,745 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:24,745 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:24,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:24,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:24,745 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:24,745 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:24,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,746 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:24,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,747 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,747 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-10-27 23:58:24,747 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:24,747 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-10-27 23:58:24,747 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:24,747 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-10-27 23:58:24,748 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:24,748 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:24,748 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:24,748 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. started... 2023-10-27 23:58:24,748 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing cb61fdd866246978279c7ac1cfc5941f 1/1 column families, dataSize=74 B heapSize=408 B 2023-10-27 23:58:25,191 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511284dd9a28f19be4eb1bca240242c917e7b to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511284dd9a28f19be4eb1bca240242c917e7b 2023-10-27 23:58:25,336 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-10-27 23:58:25,338 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-10-27 23:58:25,605 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/.tmp/family1/4e5b16202d2c4b7baf3452c0a5c79f07 2023-10-27 23:58:25,636 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/.tmp/family1/4e5b16202d2c4b7baf3452c0a5c79f07 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/4e5b16202d2c4b7baf3452c0a5c79f07 2023-10-27 23:58:25,649 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/4e5b16202d2c4b7baf3452c0a5c79f07, entries=1, sequenceid=9, filesize=5.0 K 2023-10-27 23:58:25,650 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for cb61fdd866246978279c7ac1cfc5941f in 902ms, sequenceid=9, compaction requested=false 2023-10-27 23:58:25,650 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for cb61fdd866246978279c7ac1cfc5941f: 2023-10-27 23:58:25,651 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:25,651 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:25,651 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:25,651 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:25,651 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-10-27 23:58:25,651 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-10-27 23:58:25,661 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:25,661 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:25,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:25,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:25,662 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:25,662 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:25,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:25,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:25,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:25,665 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-10-27 23:58:25,665 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@eb901ea[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:25,665 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-10-27 23:58:25,665 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,670 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,670 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,670 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-10-27 23:58:25,670 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-10-27 23:58:25,670 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:25,670 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:25,670 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:25,695 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:25,695 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-10-27 23:58:25,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:25,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:25,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:25,695 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:25,696 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-10-27 23:58:25,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:25,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:25,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:25,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:25,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:25,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:58:25,704 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-10-27 23:58:25,704 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-10-27 23:58:25,705 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:25,705 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:25,705 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,705 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:25,720 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,720 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,720 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:25,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:25,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:25,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:25,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:25,721 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:25,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:25,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:25,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,724 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:25,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:25,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:25,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:25,736 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:25,737 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,737 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,737 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:25,737 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-10-27 23:58:25,736 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:25,737 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:25,737 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,737 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-10-27 23:58:25,737 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:25,737 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:25,737 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:25,737 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:25,738 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:25,738 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:25,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:25,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:25,838 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:25,841 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:25,855 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:25,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-10-27 23:58:25,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,858 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-10-27 23:58:25,858 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:25,859 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-10-27 23:58:25,859 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-10-27 23:58:25,860 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,860 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,870 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:25,870 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:25,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:25,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:25,870 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:25,870 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:25,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,871 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:25,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,871 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,872 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-10-27 23:58:25,872 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,872 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-10-27 23:58:25,872 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:25,872 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-10-27 23:58:25,872 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:25,873 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:25,873 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:25,873 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. started... 2023-10-27 23:58:25,873 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing cb61fdd866246978279c7ac1cfc5941f 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:58:25,898 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201f219520f6448499a9da51035ad582d13 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201f219520f6448499a9da51035ad582d13 2023-10-27 23:58:25,910 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/.tmp/family1/b325face36034bbd86a60ab50cd52237 2023-10-27 23:58:25,921 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/.tmp/family1/b325face36034bbd86a60ab50cd52237 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/b325face36034bbd86a60ab50cd52237 2023-10-27 23:58:25,931 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/b325face36034bbd86a60ab50cd52237, entries=1, sequenceid=13, filesize=5.0 K 2023-10-27 23:58:25,932 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cb61fdd866246978279c7ac1cfc5941f in 59ms, sequenceid=13, compaction requested=true 2023-10-27 23:58:25,932 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for cb61fdd866246978279c7ac1cfc5941f: 2023-10-27 23:58:25,932 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:25,932 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:25,933 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:25,933 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:25,933 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-10-27 23:58:25,933 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-10-27 23:58:25,955 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,955 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:25,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:25,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:25,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:25,956 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,956 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:25,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:25,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:25,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:25,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:25,960 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-10-27 23:58:25,960 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@333b786e[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:25,960 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-10-27 23:58:25,960 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,999 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:25,999 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-10-27 23:58:25,999 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-10-27 23:58:25,999 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:25,999 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,000 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:26,020 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-10-27 23:58:26,020 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:26,021 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-10-27 23:58:26,020 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:26,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:26,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:26,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:26,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:26,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:58:26,026 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-10-27 23:58:26,026 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-10-27 23:58:26,026 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:26,026 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:26,026 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,026 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:26,037 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,037 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,037 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:26,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:26,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:26,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:26,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:26,037 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:26,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:26,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,039 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:26,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,052 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,052 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:26,052 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:26,052 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-10-27 23:58:26,052 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,052 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-10-27 23:58:26,053 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:26,052 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:26,052 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:26,052 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:26,053 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,053 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,053 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,053 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:26,054 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:26,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:26,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:26,154 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:26,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:26,165 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:26,168 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-10-27 23:58:26,168 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,168 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-10-27 23:58:26,168 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:26,169 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-10-27 23:58:26,169 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-10-27 23:58:26,169 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,169 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,178 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,178 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:26,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:26,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:26,179 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,179 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:26,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,179 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:26,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,180 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,180 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-10-27 23:58:26,180 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,180 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-10-27 23:58:26,180 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:26,181 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-10-27 23:58:26,181 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:26,181 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:26,181 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:26,181 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. started... 2023-10-27 23:58:26,181 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing cb61fdd866246978279c7ac1cfc5941f 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:58:26,612 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205fbf76c88e7d5400fb42d2f8a48a6e59e to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205fbf76c88e7d5400fb42d2f8a48a6e59e 2023-10-27 23:58:26,639 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/.tmp/family1/b08be717a92e4c8bb5a963e7d90a8bed 2023-10-27 23:58:26,658 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/.tmp/family1/b08be717a92e4c8bb5a963e7d90a8bed as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/b08be717a92e4c8bb5a963e7d90a8bed 2023-10-27 23:58:26,679 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/b08be717a92e4c8bb5a963e7d90a8bed, entries=1, sequenceid=17, filesize=5.0 K 2023-10-27 23:58:26,680 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cb61fdd866246978279c7ac1cfc5941f in 499ms, sequenceid=17, compaction requested=true 2023-10-27 23:58:26,681 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for cb61fdd866246978279c7ac1cfc5941f: 2023-10-27 23:58:26,681 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:26,681 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:26,681 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:26,681 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,682 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-10-27 23:58:26,682 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-10-27 23:58:26,712 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,712 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:26,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:26,712 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,712 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:26,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:26,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:26,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:26,715 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-10-27 23:58:26,715 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@450de7d8[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:26,715 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-10-27 23:58:26,715 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,720 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,720 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-10-27 23:58:26,720 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-10-27 23:58:26,720 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,720 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:26,720 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:26,729 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-10-27 23:58:26,729 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,729 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:26,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:26,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:26,729 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-10-27 23:58:26,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:26,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:26,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:26,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:58:26,734 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-10-27 23:58:26,734 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-10-27 23:58:26,734 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:26,734 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:26,734 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,734 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:26,746 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,746 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:26,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,746 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:26,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:26,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:26,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:26,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,748 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:26,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:26,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,750 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:26,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,762 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,762 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,762 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,762 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:26,762 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:26,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:26,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:26,762 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-10-27 23:58:26,762 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,763 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,763 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,763 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-10-27 23:58:26,763 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:26,763 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:26,763 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,763 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:26,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:26,763 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:26,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:26,864 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:26,866 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:26,876 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:26,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-10-27 23:58:26,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,879 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-10-27 23:58:26,879 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:26,880 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-10-27 23:58:26,880 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-10-27 23:58:26,881 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,881 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,887 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:26,887 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:26,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:26,887 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:26,887 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:26,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,889 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:26,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,890 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,891 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-10-27 23:58:26,891 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:26,891 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-10-27 23:58:26,891 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:26,891 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-10-27 23:58:26,892 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:26,892 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:26,892 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:26,892 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. started... 2023-10-27 23:58:26,892 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing cb61fdd866246978279c7ac1cfc5941f 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:58:27,328 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122830b6542c096348028d7a82ea48a8eee4 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015122830b6542c096348028d7a82ea48a8eee4 2023-10-27 23:58:27,342 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/.tmp/family1/4da1b63f68394f6c9acee08036f0c10e 2023-10-27 23:58:27,352 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/.tmp/family1/4da1b63f68394f6c9acee08036f0c10e as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/4da1b63f68394f6c9acee08036f0c10e 2023-10-27 23:58:27,366 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/4da1b63f68394f6c9acee08036f0c10e, entries=1, sequenceid=21, filesize=5.0 K 2023-10-27 23:58:27,367 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cb61fdd866246978279c7ac1cfc5941f in 475ms, sequenceid=21, compaction requested=true 2023-10-27 23:58:27,367 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for cb61fdd866246978279c7ac1cfc5941f: 2023-10-27 23:58:27,367 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:27,367 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:27,369 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:27,369 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:27,369 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-10-27 23:58:27,369 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-10-27 23:58:27,421 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,421 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:27,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:27,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:27,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:27,423 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,423 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:27,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:27,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:27,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:27,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:27,425 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-10-27 23:58:27,425 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@555713b1[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:27,425 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-10-27 23:58:27,425 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,437 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,437 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-10-27 23:58:27,437 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-10-27 23:58:27,437 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:27,437 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:27,437 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:27,445 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-10-27 23:58:27,445 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:27,445 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:27,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:27,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:27,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:27,445 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-10-27 23:58:27,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:27,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:27,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:27,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:27,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:27,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:58:27,450 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-10-27 23:58:27,450 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-10-27 23:58:27,450 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:27,450 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:27,450 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,450 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:27,460 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,460 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,460 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:27,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:27,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:27,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:27,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:27,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:27,461 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:27,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:27,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:27,463 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:27,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:27,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:27,470 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:27,470 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:27,470 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,470 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,470 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:27,470 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,470 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,470 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,470 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:27,470 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:27,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:27,470 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-10-27 23:58:27,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:27,471 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-10-27 23:58:27,471 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:27,472 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:27,472 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:27,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:27,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:27,572 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:27,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:27,584 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:27,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-10-27 23:58:27,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,588 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-10-27 23:58:27,588 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:27,588 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-10-27 23:58:27,589 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-10-27 23:58:27,589 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,589 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,595 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:27,595 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:27,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:27,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:27,596 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:27,596 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:27,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,596 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:27,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,597 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,597 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-10-27 23:58:27,597 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:27,597 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-10-27 23:58:27,597 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:27,598 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-10-27 23:58:27,598 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:27,598 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:27,598 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:27,598 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. started... 2023-10-27 23:58:27,598 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing cb61fdd866246978279c7ac1cfc5941f 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:58:28,028 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512318153f686dcd44d6492b7fdc458c39003 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512318153f686dcd44d6492b7fdc458c39003 2023-10-27 23:58:28,439 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/.tmp/family1/9f9f51f1601a4db3904ff4e0b50605cc 2023-10-27 23:58:28,450 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/.tmp/family1/9f9f51f1601a4db3904ff4e0b50605cc as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/9f9f51f1601a4db3904ff4e0b50605cc 2023-10-27 23:58:28,459 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/9f9f51f1601a4db3904ff4e0b50605cc, entries=1, sequenceid=25, filesize=5.0 K 2023-10-27 23:58:28,461 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cb61fdd866246978279c7ac1cfc5941f in 863ms, sequenceid=25, compaction requested=true 2023-10-27 23:58:28,461 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for cb61fdd866246978279c7ac1cfc5941f: 2023-10-27 23:58:28,461 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:28,461 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:28,462 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:28,462 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:28,462 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-10-27 23:58:28,462 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-10-27 23:58:28,495 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,495 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:28,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:28,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:28,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:28,495 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,496 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:28,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:28,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:28,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:28,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:28,498 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-10-27 23:58:28,498 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1327fea0[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:28,498 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-10-27 23:58:28,498 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,503 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,503 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-10-27 23:58:28,504 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:28,504 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:28,504 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-10-27 23:58:28,504 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:28,512 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-10-27 23:58:28,512 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:28,512 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-10-27 23:58:28,512 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:28,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:28,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:28,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:28,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:28,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:28,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:28,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:28,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:28,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:58:28,515 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-10-27 23:58:28,516 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-10-27 23:58:28,516 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:28,516 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:28,516 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,516 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:28,527 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,527 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,527 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:28,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:28,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:28,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:28,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:28,528 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:28,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:28,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:28,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,529 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:28,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:28,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:28,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:28,537 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:28,537 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:28,537 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:28,537 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-10-27 23:58:28,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:28,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:28,537 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:28,537 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-10-27 23:58:28,538 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:28,537 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,538 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,538 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:28,538 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:28,538 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,538 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:28,538 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:28,538 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:28,638 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:28,640 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:28,647 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:28,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-10-27 23:58:28,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,650 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-10-27 23:58:28,650 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:28,651 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-10-27 23:58:28,651 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-10-27 23:58:28,651 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,651 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,677 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:28,677 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:28,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:28,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:28,677 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:28,677 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:28,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,678 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:28,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,679 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,679 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-10-27 23:58:28,679 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:28,679 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-10-27 23:58:28,679 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:28,680 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-10-27 23:58:28,680 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:28,680 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:28,680 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:28,680 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. started... 2023-10-27 23:58:28,680 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing cb61fdd866246978279c7ac1cfc5941f 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:58:29,103 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101ddd5d0bfb741475f9133cd0481106ff6 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101ddd5d0bfb741475f9133cd0481106ff6 2023-10-27 23:58:29,177 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/.tmp/family1/2fac5561a35243de927ff462e1d0083e 2023-10-27 23:58:29,189 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/.tmp/family1/2fac5561a35243de927ff462e1d0083e as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/2fac5561a35243de927ff462e1d0083e 2023-10-27 23:58:29,197 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/2fac5561a35243de927ff462e1d0083e, entries=1, sequenceid=29, filesize=5.0 K 2023-10-27 23:58:29,198 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cb61fdd866246978279c7ac1cfc5941f in 518ms, sequenceid=29, compaction requested=true 2023-10-27 23:58:29,198 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for cb61fdd866246978279c7ac1cfc5941f: 2023-10-27 23:58:29,199 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:29,199 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:29,199 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:29,199 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,200 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-10-27 23:58:29,200 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-10-27 23:58:29,228 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,228 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:29,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:29,229 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,229 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:29,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:29,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:29,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:29,233 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-10-27 23:58:29,234 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b371ca2[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:29,234 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-10-27 23:58:29,234 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,244 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,244 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-10-27 23:58:29,244 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-10-27 23:58:29,244 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:29,244 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,245 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:29,253 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-10-27 23:58:29,253 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:29,253 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-10-27 23:58:29,253 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:29,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:29,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:29,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:29,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:29,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:58:29,263 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-10-27 23:58:29,263 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-10-27 23:58:29,263 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:29,263 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:29,263 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,263 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:29,270 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,270 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:29,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:29,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,270 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:29,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:29,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:29,271 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:29,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:29,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,273 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:29,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,308 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,308 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:29,308 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,308 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:29,308 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-10-27 23:58:29,309 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:29,308 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:29,308 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:29,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:29,310 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,310 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:29,310 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,310 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:29,311 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:29,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:29,311 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,411 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:29,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:29,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:29,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-10-27 23:58:29,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,425 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-10-27 23:58:29,425 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:29,426 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-10-27 23:58:29,426 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-10-27 23:58:29,426 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,427 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,436 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:29,436 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:29,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:29,437 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,437 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:29,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,438 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:29,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,438 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,439 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-10-27 23:58:29,439 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,439 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-10-27 23:58:29,439 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:29,439 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-10-27 23:58:29,440 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:29,440 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:29,440 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:29,440 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. started... 2023-10-27 23:58:29,440 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing cb61fdd866246978279c7ac1cfc5941f 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:58:29,468 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601039c45b24101f049719b88b396ad1b4987 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601039c45b24101f049719b88b396ad1b4987 2023-10-27 23:58:29,878 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/.tmp/family1/73d1117893954bf08fb5c7343cd7d722 2023-10-27 23:58:29,893 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/.tmp/family1/73d1117893954bf08fb5c7343cd7d722 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/73d1117893954bf08fb5c7343cd7d722 2023-10-27 23:58:29,902 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/73d1117893954bf08fb5c7343cd7d722, entries=1, sequenceid=33, filesize=5.0 K 2023-10-27 23:58:29,904 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cb61fdd866246978279c7ac1cfc5941f in 464ms, sequenceid=33, compaction requested=true 2023-10-27 23:58:29,904 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for cb61fdd866246978279c7ac1cfc5941f: 2023-10-27 23:58:29,904 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:29,904 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:29,904 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:29,904 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,904 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-10-27 23:58:29,904 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-10-27 23:58:29,936 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,937 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:29,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:29,937 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,937 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:29,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:29,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:29,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:29,941 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-10-27 23:58:29,941 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@512e073b[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:29,941 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-10-27 23:58:29,941 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,952 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,952 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-10-27 23:58:29,952 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-10-27 23:58:29,952 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:29,952 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,953 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:29,962 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-10-27 23:58:29,962 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:29,962 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-10-27 23:58:29,962 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:29,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:29,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:29,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:29,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:29,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:58:29,971 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-10-27 23:58:29,971 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-10-27 23:58:29,971 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:29,972 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:29,972 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,972 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:29,978 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,978 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:29,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:29,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,978 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:29,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:29,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:29,979 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:29,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:29,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,980 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:29,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,986 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,986 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:29,986 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:29,986 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:29,986 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-10-27 23:58:29,986 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-10-27 23:58:29,987 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:29,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:29,986 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,987 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:29,986 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:29,987 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,987 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,987 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:29,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:29,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:29,987 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:29,988 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:30,088 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:30,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:30,101 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:30,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-10-27 23:58:30,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:30,103 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-10-27 23:58:30,103 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:30,104 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-10-27 23:58:30,104 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-10-27 23:58:30,105 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:30,105 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:30,130 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:30,130 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:30,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:30,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:30,131 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:30,131 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:30,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:30,131 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:30,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:30,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:30,132 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:30,133 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-10-27 23:58:30,133 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:30,134 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-10-27 23:58:30,134 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:30,135 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-10-27 23:58:30,135 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:30,135 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:30,135 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:30,136 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. started... 2023-10-27 23:58:30,136 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing cb61fdd866246978279c7ac1cfc5941f 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:58:30,570 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509078d4d888e14c04051911b0abbecac1de4 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201509078d4d888e14c04051911b0abbecac1de4 2023-10-27 23:58:30,982 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/.tmp/family1/7a140ee28c6d42779f424ef69c2223f9 2023-10-27 23:58:30,997 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/.tmp/family1/7a140ee28c6d42779f424ef69c2223f9 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/7a140ee28c6d42779f424ef69c2223f9 2023-10-27 23:58:31,005 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/7a140ee28c6d42779f424ef69c2223f9, entries=1, sequenceid=37, filesize=5.0 K 2023-10-27 23:58:31,006 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cb61fdd866246978279c7ac1cfc5941f in 870ms, sequenceid=37, compaction requested=true 2023-10-27 23:58:31,006 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for cb61fdd866246978279c7ac1cfc5941f: 2023-10-27 23:58:31,006 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:31,006 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:31,007 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:31,007 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:31,007 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-10-27 23:58:31,007 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-10-27 23:58:31,036 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:31,037 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:31,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:31,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:31,037 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,037 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:31,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:31,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:31,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:31,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:31,041 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-10-27 23:58:31,041 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3c095c34[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:31,041 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-10-27 23:58:31,041 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,052 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,052 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-10-27 23:58:31,053 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-10-27 23:58:31,053 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:31,053 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:31,053 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:31,062 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-10-27 23:58:31,062 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:31,062 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-10-27 23:58:31,062 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:31,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:31,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:31,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:31,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:31,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:31,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:31,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:31,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:31,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:58:31,071 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-10-27 23:58:31,071 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-10-27 23:58:31,071 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:31,071 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:31,071 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,071 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:31,078 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,078 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,078 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:31,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:31,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:31,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:31,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:31,079 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:31,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:31,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:31,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,080 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:31,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:31,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:31,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:31,086 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:31,086 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,086 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,086 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:31,086 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-10-27 23:58:31,086 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:31,086 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-10-27 23:58:31,087 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:31,087 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,086 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:31,087 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:31,087 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:31,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:31,087 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,088 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:31,088 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:31,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:31,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:31,188 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:31,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:31,201 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:31,204 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-10-27 23:58:31,204 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,204 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-10-27 23:58:31,204 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:31,204 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-10-27 23:58:31,204 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-10-27 23:58:31,205 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,205 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,211 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:31,211 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:31,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:31,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:31,212 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:31,212 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:31,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,212 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:31,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,213 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,213 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-10-27 23:58:31,213 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:31,213 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-10-27 23:58:31,213 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:31,214 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-10-27 23:58:31,214 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:31,214 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:31,214 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:31,215 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. started... 2023-10-27 23:58:31,215 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing cb61fdd866246978279c7ac1cfc5941f 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:58:31,645 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120cadd787650734894867b43995ca6358a to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120cadd787650734894867b43995ca6358a 2023-10-27 23:58:32,055 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/.tmp/family1/464c2f618e0f46bfae198b66a0c81513 2023-10-27 23:58:32,073 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/.tmp/family1/464c2f618e0f46bfae198b66a0c81513 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/464c2f618e0f46bfae198b66a0c81513 2023-10-27 23:58:32,080 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/464c2f618e0f46bfae198b66a0c81513, entries=1, sequenceid=41, filesize=5.0 K 2023-10-27 23:58:32,081 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cb61fdd866246978279c7ac1cfc5941f in 866ms, sequenceid=41, compaction requested=true 2023-10-27 23:58:32,081 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for cb61fdd866246978279c7ac1cfc5941f: 2023-10-27 23:58:32,081 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:32,081 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:32,082 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:32,082 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:32,082 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-10-27 23:58:32,082 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-10-27 23:58:32,111 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:32,111 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:32,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:32,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:32,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:32,112 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:32,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:32,112 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:32,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:32,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:32,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:32,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:32,115 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-10-27 23:58:32,115 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@107767eb[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:32,115 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-10-27 23:58:32,115 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:32,141 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:32,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:32,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:32,142 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-10-27 23:58:32,142 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-10-27 23:58:32,142 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:32,142 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:32,142 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:32,153 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:32,153 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-10-27 23:58:32,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:32,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:32,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:32,154 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:32,154 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-10-27 23:58:32,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:32,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:32,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:32,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:32,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:32,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:32,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:32,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:58:32,164 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-10-27 23:58:32,164 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-10-27 23:58:32,164 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:32,164 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:32,164 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:32,165 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:32,170 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:32,170 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:32,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:32,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:32,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:32,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:32,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:32,170 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:32,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:32,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:32,171 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:32,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:32,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:32,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:32,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:32,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:32,172 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:32,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:32,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:32,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:32,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:32,178 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:32,178 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:32,178 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:32,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:32,178 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-10-27 23:58:32,178 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:32,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:32,178 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-10-27 23:58:32,179 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:32,178 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:32,178 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:32,179 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:32,179 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:32,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:32,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:32,179 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:32,179 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:32,179 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:32,180 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:32,280 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:32,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:32,290 DEBUG [Listener at localhost/40295] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-10-27 23:58:32,293 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:56114, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-10-27 23:58:32,295 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36413] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:32,308 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] procedure2.ProcedureExecutor(1028): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-10-27 23:58:32,309 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-10-27 23:58:32,311 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-27 23:58:32,311 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698451712310 2023-10-27 23:58:32,314 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-10-27 23:58:32,320 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-10-27 23:58:32,320 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-10-27 23:58:32,322 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] zookeeper.ReadOnlyZKClient(139): Connect 0x551ac9e9 to 127.0.0.1:57369 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-10-27 23:58:32,329 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5a7b8bc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-27 23:58:32,331 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-10-27 23:58:33,220 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103014ccf86b2e0472e9ca2abb8b7ceae4d to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103014ccf86b2e0472e9ca2abb8b7ceae4d 2023-10-27 23:58:33,222 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-10-27 23:58:33,239 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-10-27 23:58:33,240 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:46700, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-10-27 23:58:33,252 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-10-27 23:58:33,253 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:49026, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-10-27 23:58:33,290 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/09c904d52b0c4fc690b53c3c351c3af6 first=Optional[r3] last=Optional[r6] 2023-10-27 23:58:33,300 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/09c904d52b0c4fc690b53c3c351c3af6}] 2023-10-27 23:58:33,317 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/09c904d52b0c4fc690b53c3c351c3af6 for inclusion in cb61fdd866246978279c7ac1cfc5941f/family1 2023-10-27 23:58:33,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-10-27 23:58:33,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(777): Region bounds: first= last= 2023-10-27 23:58:33,328 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for cb61fdd866246978279c7ac1cfc5941f: 2023-10-27 23:58:33,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/09c904d52b0c4fc690b53c3c351c3af6 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__lk8edp0d6ald6f6jc1k1j9vl7e1sv7cpk2bnims49mjppb9p262458q7pk1melc6/family1/09c904d52b0c4fc690b53c3c351c3af6 2023-10-27 23:58:33,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__lk8edp0d6ald6f6jc1k1j9vl7e1sv7cpk2bnims49mjppb9p262458q7pk1melc6/family1/09c904d52b0c4fc690b53c3c351c3af6 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/fd75919f93f941b4966a7b6c4d469a22_SeqId_44_ 2023-10-27 23:58:33,334 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__lk8edp0d6ald6f6jc1k1j9vl7e1sv7cpk2bnims49mjppb9p262458q7pk1melc6/family1/09c904d52b0c4fc690b53c3c351c3af6 into cb61fdd866246978279c7ac1cfc5941f/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/fd75919f93f941b4966a7b6c4d469a22_SeqId_44_ - updating store file list. 2023-10-27 23:58:33,341 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/fd75919f93f941b4966a7b6c4d469a22_SeqId_44_ into cb61fdd866246978279c7ac1cfc5941f/family1 2023-10-27 23:58:33,341 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__lk8edp0d6ald6f6jc1k1j9vl7e1sv7cpk2bnims49mjppb9p262458q7pk1melc6/family1/09c904d52b0c4fc690b53c3c351c3af6 into cb61fdd866246978279c7ac1cfc5941f/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/fd75919f93f941b4966a7b6c4d469a22_SeqId_44_) 2023-10-27 23:58:33,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__lk8edp0d6ald6f6jc1k1j9vl7e1sv7cpk2bnims49mjppb9p262458q7pk1melc6/family1/09c904d52b0c4fc690b53c3c351c3af6 2023-10-27 23:58:33,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-10-27 23:58:33,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-10-27 23:58:33,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(1912): cb61fdd866246978279c7ac1cfc5941f/family1 is initiating minor compaction 2023-10-27 23:58:33,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2071f207; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-27 23:58:33,352 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of cb61fdd866246978279c7ac1cfc5941f/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:33,353 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/4e5b16202d2c4b7baf3452c0a5c79f07, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/b325face36034bbd86a60ab50cd52237, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/b08be717a92e4c8bb5a963e7d90a8bed, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/4da1b63f68394f6c9acee08036f0c10e, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/9f9f51f1601a4db3904ff4e0b50605cc, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/2fac5561a35243de927ff462e1d0083e, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/73d1117893954bf08fb5c7343cd7d722, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/7a140ee28c6d42779f424ef69c2223f9, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/464c2f618e0f46bfae198b66a0c81513, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/fd75919f93f941b4966a7b6c4d469a22_SeqId_44_] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/.tmp, totalSize=50.2 K 2023-10-27 23:58:33,354 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 4e5b16202d2c4b7baf3452c0a5c79f07, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-10-27 23:58:33,354 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting b325face36034bbd86a60ab50cd52237, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-10-27 23:58:33,354 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting b08be717a92e4c8bb5a963e7d90a8bed, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-10-27 23:58:33,354 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 4da1b63f68394f6c9acee08036f0c10e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-10-27 23:58:33,354 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 9f9f51f1601a4db3904ff4e0b50605cc, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-10-27 23:58:33,354 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 2fac5561a35243de927ff462e1d0083e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-10-27 23:58:33,354 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 73d1117893954bf08fb5c7343cd7d722, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-10-27 23:58:33,354 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 7a140ee28c6d42779f424ef69c2223f9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-10-27 23:58:33,354 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 464c2f618e0f46bfae198b66a0c81513, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-10-27 23:58:33,354 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting fd75919f93f941b4966a7b6c4d469a22_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-10-27 23:58:33,367 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-10-27 23:58:33,369 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:58:33,375 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015122830b6542c096348028d7a82ea48a8eee4 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015122830b6542c096348028d7a82ea48a8eee4 2023-10-27 23:58:33,377 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512318153f686dcd44d6492b7fdc458c39003 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512318153f686dcd44d6492b7fdc458c39003 2023-10-27 23:58:33,379 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101ddd5d0bfb741475f9133cd0481106ff6 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101ddd5d0bfb741475f9133cd0481106ff6 2023-10-27 23:58:33,380 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601039c45b24101f049719b88b396ad1b4987 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601039c45b24101f049719b88b396ad1b4987 2023-10-27 23:58:33,381 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-10-27 23:58:33,381 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-10-27 23:58:33,381 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-10-27 23:58:33,381 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-10-27 23:58:33,382 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-10-27 23:58:33,382 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-10-27 23:58:33,382 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-10-27 23:58:33,382 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-10-27 23:58:33,382 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-10-27 23:58:33,382 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-10-27 23:58:33,409 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] throttle.PressureAwareThroughputController(145): cb61fdd866246978279c7ac1cfc5941f#family1#compaction#10 average throughput is 0.24 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-10-27 23:58:33,841 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/.tmp/family1/7e92e4c8fd644c6a98044dc59beea7ba as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/7e92e4c8fd644c6a98044dc59beea7ba 2023-10-27 23:58:33,857 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in cb61fdd866246978279c7ac1cfc5941f/family1 of cb61fdd866246978279c7ac1cfc5941f into 7e92e4c8fd644c6a98044dc59beea7ba(size=6.6 K), total size for store is 182.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-10-27 23:58:33,857 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for cb61fdd866246978279c7ac1cfc5941f: 2023-10-27 23:58:33,858 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1698451113350; duration=0sec 2023-10-27 23:58:33,859 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-27 23:58:33,947 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/4e5b16202d2c4b7baf3452c0a5c79f07, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/b325face36034bbd86a60ab50cd52237, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/b08be717a92e4c8bb5a963e7d90a8bed, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/4da1b63f68394f6c9acee08036f0c10e, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/9f9f51f1601a4db3904ff4e0b50605cc, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/2fac5561a35243de927ff462e1d0083e, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/73d1117893954bf08fb5c7343cd7d722, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/7a140ee28c6d42779f424ef69c2223f9, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/464c2f618e0f46bfae198b66a0c81513, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/fd75919f93f941b4966a7b6c4d469a22_SeqId_44_] to archive 2023-10-27 23:58:33,948 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:58:33,951 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/4e5b16202d2c4b7baf3452c0a5c79f07 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/4e5b16202d2c4b7baf3452c0a5c79f07 2023-10-27 23:58:33,953 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/b325face36034bbd86a60ab50cd52237 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/b325face36034bbd86a60ab50cd52237 2023-10-27 23:58:33,954 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/b08be717a92e4c8bb5a963e7d90a8bed to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/b08be717a92e4c8bb5a963e7d90a8bed 2023-10-27 23:58:33,956 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/4da1b63f68394f6c9acee08036f0c10e to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/4da1b63f68394f6c9acee08036f0c10e 2023-10-27 23:58:33,958 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/9f9f51f1601a4db3904ff4e0b50605cc to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/9f9f51f1601a4db3904ff4e0b50605cc 2023-10-27 23:58:33,959 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/2fac5561a35243de927ff462e1d0083e to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/2fac5561a35243de927ff462e1d0083e 2023-10-27 23:58:33,961 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/73d1117893954bf08fb5c7343cd7d722 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/73d1117893954bf08fb5c7343cd7d722 2023-10-27 23:58:33,963 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/7a140ee28c6d42779f424ef69c2223f9 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/7a140ee28c6d42779f424ef69c2223f9 2023-10-27 23:58:33,964 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/464c2f618e0f46bfae198b66a0c81513 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/464c2f618e0f46bfae198b66a0c81513 2023-10-27 23:58:33,966 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/fd75919f93f941b4966a7b6c4d469a22_SeqId_44_ to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/fd75919f93f941b4966a7b6c4d469a22_SeqId_44_ 2023-10-27 23:58:34,287 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512053e774156409740fbbe089327d5e97700 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512053e774156409740fbbe089327d5e97700 2023-10-27 23:58:34,289 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-10-27 23:58:34,319 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b505a68539b247559f36835cdde803b7 first=Optional[r00] last=Optional[r2] 2023-10-27 23:58:34,321 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b505a68539b247559f36835cdde803b7}] 2023-10-27 23:58:34,323 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b505a68539b247559f36835cdde803b7 for inclusion in cb61fdd866246978279c7ac1cfc5941f/family1 2023-10-27 23:58:34,328 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-10-27 23:58:34,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(777): Region bounds: first= last= 2023-10-27 23:58:34,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for cb61fdd866246978279c7ac1cfc5941f: 2023-10-27 23:58:34,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b505a68539b247559f36835cdde803b7 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__858llqjgl46lck2qith3bs17m1shdu9ci8in0qb8gc86l8pf98k880r8fuqsjtv7/family1/b505a68539b247559f36835cdde803b7 2023-10-27 23:58:34,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__858llqjgl46lck2qith3bs17m1shdu9ci8in0qb8gc86l8pf98k880r8fuqsjtv7/family1/b505a68539b247559f36835cdde803b7 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/0b1cd3983b3849168bff92ff1c186237_SeqId_47_ 2023-10-27 23:58:34,333 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__858llqjgl46lck2qith3bs17m1shdu9ci8in0qb8gc86l8pf98k880r8fuqsjtv7/family1/b505a68539b247559f36835cdde803b7 into cb61fdd866246978279c7ac1cfc5941f/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/0b1cd3983b3849168bff92ff1c186237_SeqId_47_ - updating store file list. 2023-10-27 23:58:34,341 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/0b1cd3983b3849168bff92ff1c186237_SeqId_47_ into cb61fdd866246978279c7ac1cfc5941f/family1 2023-10-27 23:58:34,341 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__858llqjgl46lck2qith3bs17m1shdu9ci8in0qb8gc86l8pf98k880r8fuqsjtv7/family1/b505a68539b247559f36835cdde803b7 into cb61fdd866246978279c7ac1cfc5941f/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/0b1cd3983b3849168bff92ff1c186237_SeqId_47_) 2023-10-27 23:58:34,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__858llqjgl46lck2qith3bs17m1shdu9ci8in0qb8gc86l8pf98k880r8fuqsjtv7/family1/b505a68539b247559f36835cdde803b7 2023-10-27 23:58:34,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-10-27 23:58:34,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-10-27 23:58:34,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(1912): cb61fdd866246978279c7ac1cfc5941f/family1 is initiating minor compaction (all files) 2023-10-27 23:58:34,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@78fd40b1; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-27 23:58:34,344 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of cb61fdd866246978279c7ac1cfc5941f/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:34,345 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/ab74b3c95daf4454b233f8a94d42f14e, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/7e92e4c8fd644c6a98044dc59beea7ba, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/0b1cd3983b3849168bff92ff1c186237_SeqId_47_] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/.tmp, totalSize=358.6 K 2023-10-27 23:58:34,345 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting ab74b3c95daf4454b233f8a94d42f14e, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878821089 2023-10-27 23:58:34,346 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 7e92e4c8fd644c6a98044dc59beea7ba, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621221089 2023-10-27 23:58:34,347 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 0b1cd3983b3849168bff92ff1c186237_SeqId_47_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878821089 2023-10-27 23:58:34,347 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-10-27 23:58:34,348 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:58:34,351 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511307a69e454f0c34748a03c93435d80400f to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511307a69e454f0c34748a03c93435d80400f 2023-10-27 23:58:34,353 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201f219520f6448499a9da51035ad582d13 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201f219520f6448499a9da51035ad582d13 2023-10-27 23:58:34,354 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205fbf76c88e7d5400fb42d2f8a48a6e59e to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205fbf76c88e7d5400fb42d2f8a48a6e59e 2023-10-27 23:58:34,355 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-10-27 23:58:34,355 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-10-27 23:58:34,355 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-10-27 23:58:34,355 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-10-27 23:58:34,356 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-27 23:58:34,357 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-10-27 23:58:34,359 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 2.0560 sec 2023-10-27 23:58:34,395 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] throttle.PressureAwareThroughputController(145): cb61fdd866246978279c7ac1cfc5941f#family1#compaction#11 average throughput is 5.01 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-10-27 23:58:34,528 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.HMaster$12(2454): Client=jenkins//188.40.62.62 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-10-27 23:58:34,537 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure2.ProcedureExecutor(1028): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:34,954 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-10-27 23:58:34,957 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-10-27 23:58:34,957 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-10-27 23:58:34,964 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-10-27 23:58:34,977 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=cb61fdd866246978279c7ac1cfc5941f, REOPEN/MOVE}] 2023-10-27 23:58:34,978 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=cb61fdd866246978279c7ac1cfc5941f, REOPEN/MOVE 2023-10-27 23:58:34,979 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=cb61fdd866246978279c7ac1cfc5941f, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:34,979 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1698451114979"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451114979"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451114979"}]},"ts":"1698451114979"} 2023-10-27 23:58:34,981 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure cb61fdd866246978279c7ac1cfc5941f, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-27 23:58:35,141 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:35,144 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(111): Close cb61fdd866246978279c7ac1cfc5941f 2023-10-27 23:58:35,144 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing cb61fdd866246978279c7ac1cfc5941f, disabling compactions & flushes 2023-10-27 23:58:35,144 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:35,633 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/.tmp/family1/fea3f1049655451580ab4716094bb657 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/fea3f1049655451580ab4716094bb657 2023-10-27 23:58:35,642 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in cb61fdd866246978279c7ac1cfc5941f/family1 of cb61fdd866246978279c7ac1cfc5941f into fea3f1049655451580ab4716094bb657(size=177.2 K), total size for store is 177.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-10-27 23:58:35,642 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for cb61fdd866246978279c7ac1cfc5941f: 2023-10-27 23:58:35,642 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:35,642 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:35,642 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f., storeName=family1, fileCount=3, fileSize=358.6 K (175.8 K, 6.6 K, 176.2 K), priority=2, time=1698451114344; duration=1sec 2023-10-27 23:58:35,643 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. after waiting 0 ms 2023-10-27 23:58:35,643 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-27 23:58:35,643 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:35,646 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/ab74b3c95daf4454b233f8a94d42f14e, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/7e92e4c8fd644c6a98044dc59beea7ba, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/0b1cd3983b3849168bff92ff1c186237_SeqId_47_] to archive 2023-10-27 23:58:35,647 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:58:35,649 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/ab74b3c95daf4454b233f8a94d42f14e to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/ab74b3c95daf4454b233f8a94d42f14e 2023-10-27 23:58:35,651 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/7e92e4c8fd644c6a98044dc59beea7ba to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/7e92e4c8fd644c6a98044dc59beea7ba 2023-10-27 23:58:35,652 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/0b1cd3983b3849168bff92ff1c186237_SeqId_47_ to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/0b1cd3983b3849168bff92ff1c186237_SeqId_47_ 2023-10-27 23:58:35,659 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-10-27 23:58:35,661 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:35,661 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for cb61fdd866246978279c7ac1cfc5941f: 2023-10-27 23:58:35,661 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(3510): Not adding moved region record: cb61fdd866246978279c7ac1cfc5941f to self. 2023-10-27 23:58:35,663 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(149): Closed cb61fdd866246978279c7ac1cfc5941f 2023-10-27 23:58:35,664 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=cb61fdd866246978279c7ac1cfc5941f, regionState=CLOSED 2023-10-27 23:58:35,664 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1698451115664"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451115664"}]},"ts":"1698451115664"} 2023-10-27 23:58:35,668 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=16, resume processing ppid=15 2023-10-27 23:58:35,668 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure cb61fdd866246978279c7ac1cfc5941f, server=jenkins-hbase16.apache.org,43469,1698451096272 in 685 msec 2023-10-27 23:58:35,669 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=cb61fdd866246978279c7ac1cfc5941f, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase16.apache.org,43469,1698451096272; forceNewPlan=false, retain=true 2023-10-27 23:58:35,820 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=cb61fdd866246978279c7ac1cfc5941f, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:35,820 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1698451115820"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451115820"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451115820"}]},"ts":"1698451115820"} 2023-10-27 23:58:35,824 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure cb61fdd866246978279c7ac1cfc5941f, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-27 23:58:35,978 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:35,981 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:35,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => cb61fdd866246978279c7ac1cfc5941f, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f.', STARTKEY => '', ENDKEY => ''} 2023-10-27 23:58:35,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly cb61fdd866246978279c7ac1cfc5941f 2023-10-27 23:58:35,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:58:35,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for cb61fdd866246978279c7ac1cfc5941f 2023-10-27 23:58:35,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for cb61fdd866246978279c7ac1cfc5941f 2023-10-27 23:58:35,984 INFO [StoreOpener-cb61fdd866246978279c7ac1cfc5941f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region cb61fdd866246978279c7ac1cfc5941f 2023-10-27 23:58:35,985 DEBUG [StoreOpener-cb61fdd866246978279c7ac1cfc5941f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1 2023-10-27 23:58:35,985 DEBUG [StoreOpener-cb61fdd866246978279c7ac1cfc5941f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1 2023-10-27 23:58:35,986 INFO [StoreOpener-cb61fdd866246978279c7ac1cfc5941f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cb61fdd866246978279c7ac1cfc5941f columnFamilyName family1 2023-10-27 23:58:35,996 DEBUG [StoreOpener-cb61fdd866246978279c7ac1cfc5941f-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/fea3f1049655451580ab4716094bb657 2023-10-27 23:58:35,997 INFO [StoreOpener-cb61fdd866246978279c7ac1cfc5941f-1] regionserver.HStore(310): Store=cb61fdd866246978279c7ac1cfc5941f/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-27 23:58:35,998 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f 2023-10-27 23:58:36,001 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f 2023-10-27 23:58:36,005 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for cb61fdd866246978279c7ac1cfc5941f 2023-10-27 23:58:36,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened cb61fdd866246978279c7ac1cfc5941f; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61480930, jitterRate=-0.08386275172233582}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-10-27 23:58:36,006 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for cb61fdd866246978279c7ac1cfc5941f: 2023-10-27 23:58:36,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f., pid=17, masterSystemTime=1698451115978 2023-10-27 23:58:36,009 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:36,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-27 23:58:36,009 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=cb61fdd866246978279c7ac1cfc5941f, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:36,010 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1698451116009"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451116009"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451116009"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451116009"}]},"ts":"1698451116009"} 2023-10-27 23:58:36,013 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=17, resume processing ppid=15 2023-10-27 23:58:36,014 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure cb61fdd866246978279c7ac1cfc5941f, server=jenkins-hbase16.apache.org,43469,1698451096272 in 188 msec 2023-10-27 23:58:36,016 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=15, resume processing ppid=14 2023-10-27 23:58:36,016 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=cb61fdd866246978279c7ac1cfc5941f, REOPEN/MOVE in 1.0370 sec 2023-10-27 23:58:36,020 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=14, resume processing ppid=13 2023-10-27 23:58:36,020 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.0550 sec 2023-10-27 23:58:36,022 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.4900 sec 2023-10-27 23:58:36,025 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=13 2023-10-27 23:58:36,032 INFO [Listener at localhost/40295] compactions.TestMobCompactor(278): alter status finished 2023-10-27 23:58:36,033 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36413] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-10-27 23:58:36,035 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] procedure2.ProcedureExecutor(1028): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-10-27 23:58:36,036 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-10-27 23:58:36,037 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-27 23:58:36,037 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698451716037 2023-10-27 23:58:36,038 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-10-27 23:58:36,040 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-10-27 23:58:36,040 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-10-27 23:58:36,042 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] zookeeper.ReadOnlyZKClient(139): Connect 0x27ac06ae to 127.0.0.1:57369 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-10-27 23:58:36,079 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7dad008c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-27 23:58:36,079 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-10-27 23:58:36,080 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-10-27 23:58:36,080 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-10-27 23:58:36,080 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-10-27 23:58:36,080 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-10-27 23:58:36,524 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128408ef8a2086440dfb61c8ff5fb45ef9b to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128408ef8a2086440dfb61c8ff5fb45ef9b 2023-10-27 23:58:36,526 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-10-27 23:58:36,541 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-10-27 23:58:36,542 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:46710, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-10-27 23:58:36,551 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-10-27 23:58:36,553 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:49032, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-10-27 23:58:36,566 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/6c6c1d7139a74c4aa1b66e87c3029dbf first=Optional[r05] last=Optional[r8] 2023-10-27 23:58:36,567 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=51 for row with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/6c6c1d7139a74c4aa1b66e87c3029dbf}] 2023-10-27 23:58:36,573 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/6c6c1d7139a74c4aa1b66e87c3029dbf for inclusion in cb61fdd866246978279c7ac1cfc5941f/family1 2023-10-27 23:58:36,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-10-27 23:58:36,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(777): Region bounds: first= last= 2023-10-27 23:58:36,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for cb61fdd866246978279c7ac1cfc5941f: 2023-10-27 23:58:36,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/6c6c1d7139a74c4aa1b66e87c3029dbf to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__7qoe9ufub6pt6qs1d6sje112n8vd445ovkemo40pdo3ho4habsoun036ukq0qv8e/family1/6c6c1d7139a74c4aa1b66e87c3029dbf 2023-10-27 23:58:36,581 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__7qoe9ufub6pt6qs1d6sje112n8vd445ovkemo40pdo3ho4habsoun036ukq0qv8e/family1/6c6c1d7139a74c4aa1b66e87c3029dbf as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/abad78ece2594e54bad454a46d7a20be_SeqId_53_ 2023-10-27 23:58:36,582 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__7qoe9ufub6pt6qs1d6sje112n8vd445ovkemo40pdo3ho4habsoun036ukq0qv8e/family1/6c6c1d7139a74c4aa1b66e87c3029dbf into cb61fdd866246978279c7ac1cfc5941f/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/abad78ece2594e54bad454a46d7a20be_SeqId_53_ - updating store file list. 2023-10-27 23:58:36,587 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/abad78ece2594e54bad454a46d7a20be_SeqId_53_ into cb61fdd866246978279c7ac1cfc5941f/family1 2023-10-27 23:58:36,587 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__7qoe9ufub6pt6qs1d6sje112n8vd445ovkemo40pdo3ho4habsoun036ukq0qv8e/family1/6c6c1d7139a74c4aa1b66e87c3029dbf into cb61fdd866246978279c7ac1cfc5941f/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/family1/abad78ece2594e54bad454a46d7a20be_SeqId_53_) 2023-10-27 23:58:36,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__7qoe9ufub6pt6qs1d6sje112n8vd445ovkemo40pdo3ho4habsoun036ukq0qv8e/family1/6c6c1d7139a74c4aa1b66e87c3029dbf 2023-10-27 23:58:36,590 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-10-27 23:58:36,591 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:58:36,592 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120cadd787650734894867b43995ca6358a to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120cadd787650734894867b43995ca6358a 2023-10-27 23:58:36,593 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511284dd9a28f19be4eb1bca240242c917e7b to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511284dd9a28f19be4eb1bca240242c917e7b 2023-10-27 23:58:36,594 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-10-27 23:58:36,594 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-10-27 23:58:36,594 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-10-27 23:58:36,594 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-10-27 23:58:36,594 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-10-27 23:58:36,594 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-10-27 23:58:36,594 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-27 23:58:36,595 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-10-27 23:58:36,598 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 562 msec 2023-10-27 23:58:36,687 INFO [Listener at localhost/40295] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=249 (was 213) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:59146 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:59048 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:47388 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:59092 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:47446 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase16:43469-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:59028 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/cluster_1ba21e36-f5ca-27f1-0277-ca251e3b747a/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:47368 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6e55ed90-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:59078 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:47462 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:47436 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x53088b12-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:59084 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:59050 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57369@0x551ac9e9-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:59152 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x53088b12-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57369@0x551ac9e9-SendThread(127.0.0.1:57369) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57369@0x551ac9e9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$46/154844478.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57369@0x27ac06ae sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$46/154844478.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-747715023_17 at /127.0.0.1:59032 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x53088b12-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57369@0x27ac06ae-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6e55ed90-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57369@0x27ac06ae-SendThread(127.0.0.1:57369) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:47426 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=591 (was 478) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=183 (was 227), ProcessCount=167 (was 167), AvailableMemoryMB=5029 (was 5165) 2023-10-27 23:58:36,697 INFO [Listener at localhost/40295] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=249, OpenFileDescriptor=591, MaxFileDescriptor=60000, SystemLoadAverage=183, ProcessCount=167, AvailableMemoryMB=5029 2023-10-27 23:58:36,700 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.HMaster$4(2112): Client=jenkins//188.40.62.62 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-10-27 23:58:36,701 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure2.ProcedureExecutor(1028): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:36,703 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-10-27 23:58:36,703 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(700): Client=jenkins//188.40.62.62 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-10-27 23:58:36,704 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-10-27 23:58:36,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-10-27 23:58:36,706 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d 2023-10-27 23:58:36,706 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d empty. 2023-10-27 23:58:36,707 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d 2023-10-27 23:58:36,707 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-10-27 23:58:36,806 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-10-27 23:58:37,008 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-10-27 23:58:37,117 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-10-27 23:58:37,119 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 1591472e0024853e35a0606e0078390d, NAME => 'testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp 2023-10-27 23:58:37,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-10-27 23:58:37,531 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:58:37,532 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing 1591472e0024853e35a0606e0078390d, disabling compactions & flushes 2023-10-27 23:58:37,532 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:37,532 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:37,532 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. after waiting 0 ms 2023-10-27 23:58:37,532 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:37,533 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:37,533 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 1591472e0024853e35a0606e0078390d: 2023-10-27 23:58:37,547 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-10-27 23:58:37,550 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1698451117549"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451117549"}]},"ts":"1698451117549"} 2023-10-27 23:58:37,554 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-10-27 23:58:37,556 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-10-27 23:58:37,556 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451117556"}]},"ts":"1698451117556"} 2023-10-27 23:58:37,558 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-10-27 23:58:37,608 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=1591472e0024853e35a0606e0078390d, ASSIGN}] 2023-10-27 23:58:37,611 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=1591472e0024853e35a0606e0078390d, ASSIGN 2023-10-27 23:58:37,613 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=1591472e0024853e35a0606e0078390d, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,43469,1698451096272; forceNewPlan=false, retain=false 2023-10-27 23:58:37,765 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=1591472e0024853e35a0606e0078390d, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:37,765 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1698451117765"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451117765"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451117765"}]},"ts":"1698451117765"} 2023-10-27 23:58:37,770 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 1591472e0024853e35a0606e0078390d, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-27 23:58:37,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-10-27 23:58:37,923 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:37,932 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:37,932 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1591472e0024853e35a0606e0078390d, NAME => 'testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d.', STARTKEY => '', ENDKEY => ''} 2023-10-27 23:58:37,933 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy 1591472e0024853e35a0606e0078390d 2023-10-27 23:58:37,933 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:58:37,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for 1591472e0024853e35a0606e0078390d 2023-10-27 23:58:37,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for 1591472e0024853e35a0606e0078390d 2023-10-27 23:58:37,937 INFO [StoreOpener-1591472e0024853e35a0606e0078390d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1591472e0024853e35a0606e0078390d 2023-10-27 23:58:37,939 DEBUG [StoreOpener-1591472e0024853e35a0606e0078390d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1 2023-10-27 23:58:37,939 DEBUG [StoreOpener-1591472e0024853e35a0606e0078390d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1 2023-10-27 23:58:37,940 INFO [StoreOpener-1591472e0024853e35a0606e0078390d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1591472e0024853e35a0606e0078390d columnFamilyName family1 2023-10-27 23:58:37,942 INFO [StoreOpener-1591472e0024853e35a0606e0078390d-1] regionserver.HStore(310): Store=1591472e0024853e35a0606e0078390d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-27 23:58:37,943 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d 2023-10-27 23:58:37,943 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d 2023-10-27 23:58:37,947 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for 1591472e0024853e35a0606e0078390d 2023-10-27 23:58:37,949 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-10-27 23:58:37,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened 1591472e0024853e35a0606e0078390d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75266887, jitterRate=0.12156401574611664}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-10-27 23:58:37,950 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for 1591472e0024853e35a0606e0078390d: 2023-10-27 23:58:37,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d., pid=21, masterSystemTime=1698451117923 2023-10-27 23:58:37,954 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:37,954 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:37,954 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=1591472e0024853e35a0606e0078390d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:37,955 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1698451117954"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451117954"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451117954"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451117954"}]},"ts":"1698451117954"} 2023-10-27 23:58:37,960 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=21, resume processing ppid=20 2023-10-27 23:58:37,960 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 1591472e0024853e35a0606e0078390d, server=jenkins-hbase16.apache.org,43469,1698451096272 in 187 msec 2023-10-27 23:58:37,963 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=20, resume processing ppid=19 2023-10-27 23:58:37,963 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=1591472e0024853e35a0606e0078390d, ASSIGN in 353 msec 2023-10-27 23:58:37,964 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-10-27 23:58:37,964 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451117964"}]},"ts":"1698451117964"} 2023-10-27 23:58:37,966 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-10-27 23:58:37,987 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-10-27 23:58:37,989 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 1.2870 sec 2023-10-27 23:58:38,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-10-27 23:58:38,816 INFO [Listener at localhost/40295] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-10-27 23:58:38,852 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:38,854 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:38,854 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-10-27 23:58:38,854 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:38,855 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-10-27 23:58:38,855 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-10-27 23:58:38,855 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:38,855 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:38,886 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:38,886 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:38,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:38,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:38,886 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:38,886 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:38,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:38,887 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:38,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:38,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:38,888 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:38,888 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:38,888 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-10-27 23:58:38,888 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:38,888 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-10-27 23:58:38,888 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:38,888 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:38,889 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:38,889 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. started... 2023-10-27 23:58:38,889 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1591472e0024853e35a0606e0078390d 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-10-27 23:58:39,316 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511309c4b73e28ec045b49348401231ab00dd to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511309c4b73e28ec045b49348401231ab00dd 2023-10-27 23:58:39,720 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/.tmp/family1/4e2dad885bee4440babf05a73e3a80d6 2023-10-27 23:58:39,732 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/.tmp/family1/4e2dad885bee4440babf05a73e3a80d6 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/4e2dad885bee4440babf05a73e3a80d6 2023-10-27 23:58:39,738 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/4e2dad885bee4440babf05a73e3a80d6, entries=1000, sequenceid=5, filesize=166.0 K 2023-10-27 23:58:39,739 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 1591472e0024853e35a0606e0078390d in 850ms, sequenceid=5, compaction requested=false 2023-10-27 23:58:39,739 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-10-27 23:58:39,740 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1591472e0024853e35a0606e0078390d: 2023-10-27 23:58:39,740 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:39,740 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:39,740 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:39,740 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:39,740 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-10-27 23:58:39,740 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-10-27 23:58:39,769 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,769 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:39,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:39,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:39,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:39,770 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,770 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:39,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:39,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:39,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:39,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:39,772 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-10-27 23:58:39,772 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5366bb4b[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:39,772 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-10-27 23:58:39,772 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,778 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,778 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-10-27 23:58:39,778 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-10-27 23:58:39,778 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:39,778 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:39,778 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:39,786 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:39,786 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-10-27 23:58:39,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:39,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:39,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:39,786 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:39,786 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-10-27 23:58:39,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:39,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:39,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:39,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:39,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:39,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:58:39,789 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:58:39,789 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-10-27 23:58:39,789 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:39,789 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:39,789 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,789 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:39,794 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,794 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:39,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:39,794 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:39,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,795 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:39,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:39,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:39,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:39,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:39,796 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:39,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:39,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:39,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:39,803 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:39,803 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:39,803 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:39,803 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:39,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:39,803 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-10-27 23:58:39,804 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:39,803 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,803 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:39,804 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:39,804 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,804 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:39,804 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-10-27 23:58:39,804 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,804 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:39,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:39,804 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:39,904 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:39,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:39,916 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:39,919 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-10-27 23:58:39,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,920 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-10-27 23:58:39,920 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:39,921 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-10-27 23:58:39,921 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-10-27 23:58:39,922 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,922 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,928 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:39,928 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:39,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:39,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:39,929 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:39,929 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:39,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,930 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:39,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,930 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,931 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-10-27 23:58:39,931 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:39,931 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-10-27 23:58:39,931 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:39,932 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-10-27 23:58:39,932 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:39,932 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:39,932 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:39,932 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. started... 2023-10-27 23:58:39,932 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1591472e0024853e35a0606e0078390d 1/1 column families, dataSize=74 B heapSize=408 B 2023-10-27 23:58:40,361 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112896a9a7bd579f4c0c8977379aa24666ce to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112896a9a7bd579f4c0c8977379aa24666ce 2023-10-27 23:58:40,769 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/.tmp/family1/94d41279f5de4ddda3c43d64f11d7b20 2023-10-27 23:58:40,779 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/.tmp/family1/94d41279f5de4ddda3c43d64f11d7b20 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/94d41279f5de4ddda3c43d64f11d7b20 2023-10-27 23:58:40,786 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/94d41279f5de4ddda3c43d64f11d7b20, entries=1, sequenceid=9, filesize=5.0 K 2023-10-27 23:58:40,787 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 1591472e0024853e35a0606e0078390d in 855ms, sequenceid=9, compaction requested=false 2023-10-27 23:58:40,787 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1591472e0024853e35a0606e0078390d: 2023-10-27 23:58:40,787 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:40,787 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:40,787 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:40,787 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:40,787 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-10-27 23:58:40,787 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-10-27 23:58:40,794 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:40,794 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:40,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:40,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:40,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:40,795 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:40,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:40,795 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:40,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:40,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:40,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:40,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:40,797 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-10-27 23:58:40,797 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@725069c3[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:40,797 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-10-27 23:58:40,798 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:40,803 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:40,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:40,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:40,803 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-10-27 23:58:40,803 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-10-27 23:58:40,803 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:40,803 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:40,803 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:40,811 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:40,811 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-10-27 23:58:40,811 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:40,811 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-10-27 23:58:40,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:40,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:40,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:40,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:40,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:40,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:40,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:40,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:40,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:40,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:40,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:58:40,819 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:58:40,819 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-10-27 23:58:40,819 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:40,819 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:40,819 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:40,819 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:40,828 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:40,828 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:40,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:40,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:40,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:40,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:40,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:40,828 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:40,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:40,829 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:40,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:40,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:40,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:40,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:40,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:40,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:40,832 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:40,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:40,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:40,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:40,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:40,878 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:40,878 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:40,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:40,878 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:40,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:40,878 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-10-27 23:58:40,879 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:40,878 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-10-27 23:58:40,878 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:40,878 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:40,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:40,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:40,879 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:40,880 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:40,879 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:40,880 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:40,880 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:40,880 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:40,880 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:40,980 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:40,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:40,992 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:40,995 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-10-27 23:58:40,995 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:40,995 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-10-27 23:58:40,996 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:40,996 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-10-27 23:58:40,996 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-10-27 23:58:40,997 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:40,997 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,011 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:41,011 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:41,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:41,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:41,012 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:41,012 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:41,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,014 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:41,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,015 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,015 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-10-27 23:58:41,015 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,016 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-10-27 23:58:41,016 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:41,017 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-10-27 23:58:41,017 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:41,017 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:41,017 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:41,017 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. started... 2023-10-27 23:58:41,018 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1591472e0024853e35a0606e0078390d 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:58:41,197 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-10-27 23:58:41,443 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201f53d6c451537482b96f1bd03760f6dad to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201f53d6c451537482b96f1bd03760f6dad 2023-10-27 23:58:41,851 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/.tmp/family1/75a0dbe8cf6e4e50a74f7ac89eda1f55 2023-10-27 23:58:41,866 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/.tmp/family1/75a0dbe8cf6e4e50a74f7ac89eda1f55 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/75a0dbe8cf6e4e50a74f7ac89eda1f55 2023-10-27 23:58:41,874 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/75a0dbe8cf6e4e50a74f7ac89eda1f55, entries=1, sequenceid=13, filesize=4.9 K 2023-10-27 23:58:41,875 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1591472e0024853e35a0606e0078390d in 857ms, sequenceid=13, compaction requested=true 2023-10-27 23:58:41,875 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1591472e0024853e35a0606e0078390d: 2023-10-27 23:58:41,875 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:41,875 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:41,875 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:41,875 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:41,875 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-10-27 23:58:41,875 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-10-27 23:58:41,911 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:41,911 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:41,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:41,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:41,912 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,912 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:41,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:41,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:41,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:41,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:41,915 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-10-27 23:58:41,915 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@44e6341[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:41,915 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-10-27 23:58:41,915 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,916 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-10-27 23:58:41,919 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,920 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-10-27 23:58:41,920 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-10-27 23:58:41,920 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:41,920 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:41,920 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:41,928 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-10-27 23:58:41,928 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:41,928 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-10-27 23:58:41,928 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:41,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:41,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:41,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:41,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:41,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:41,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:41,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:41,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:41,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:58:41,936 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:58:41,936 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-10-27 23:58:41,937 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:41,937 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:41,937 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,937 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:41,944 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,944 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:41,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:41,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,945 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:41,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:41,946 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:41,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:41,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:41,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:41,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:41,950 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:41,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:41,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:41,961 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:41,961 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:41,961 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,961 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:41,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:41,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:41,961 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:41,961 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-10-27 23:58:41,961 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-10-27 23:58:41,962 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:41,961 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,962 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:41,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:41,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:41,962 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,962 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,962 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:41,962 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:41,962 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:42,063 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:42,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:42,074 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:42,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-10-27 23:58:42,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,076 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-10-27 23:58:42,076 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:42,077 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-10-27 23:58:42,077 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-10-27 23:58:42,077 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,077 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,086 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:42,086 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:42,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:42,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:42,086 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:42,086 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:42,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,087 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:42,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,087 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,087 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-10-27 23:58:42,087 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,087 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-10-27 23:58:42,087 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:42,088 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-10-27 23:58:42,088 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:42,088 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:42,088 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:42,088 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. started... 2023-10-27 23:58:42,088 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1591472e0024853e35a0606e0078390d 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:58:42,103 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205877f4a2801bc4241a7701c02e0680943 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205877f4a2801bc4241a7701c02e0680943 2023-10-27 23:58:42,110 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/.tmp/family1/faa02e6bf99f4ba7a9fbc153f8f4dce9 2023-10-27 23:58:42,116 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/.tmp/family1/faa02e6bf99f4ba7a9fbc153f8f4dce9 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/faa02e6bf99f4ba7a9fbc153f8f4dce9 2023-10-27 23:58:42,122 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/faa02e6bf99f4ba7a9fbc153f8f4dce9, entries=1, sequenceid=17, filesize=4.9 K 2023-10-27 23:58:42,123 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1591472e0024853e35a0606e0078390d in 35ms, sequenceid=17, compaction requested=true 2023-10-27 23:58:42,123 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1591472e0024853e35a0606e0078390d: 2023-10-27 23:58:42,123 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:42,123 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:42,123 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:42,123 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:42,124 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-10-27 23:58:42,124 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-10-27 23:58:42,134 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,135 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:42,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:42,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:42,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:42,135 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,135 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:42,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:42,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:42,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:42,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:42,137 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-10-27 23:58:42,137 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2fad141c[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:42,137 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-10-27 23:58:42,137 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,144 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,144 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-10-27 23:58:42,145 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-10-27 23:58:42,145 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:42,145 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:42,145 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:42,153 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:42,153 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-10-27 23:58:42,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:42,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:42,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:42,153 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:42,153 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-10-27 23:58:42,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:42,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:42,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:42,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:42,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:42,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:58:42,157 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:58:42,157 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-10-27 23:58:42,157 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:42,158 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:42,158 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,158 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:42,168 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,168 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,168 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:42,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:42,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:42,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:42,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:42,170 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:42,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:42,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:42,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,173 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:42,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:42,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:42,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:42,186 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:42,186 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:42,186 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,186 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-10-27 23:58:42,186 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:42,186 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,186 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:42,186 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:42,186 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:42,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:42,186 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:42,187 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:42,187 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:42,187 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,187 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:42,187 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:42,287 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:42,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:42,294 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:42,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-10-27 23:58:42,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,296 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-10-27 23:58:42,296 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:42,297 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-10-27 23:58:42,297 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-10-27 23:58:42,297 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,297 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,302 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:42,303 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:42,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:42,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:42,303 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:42,303 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:42,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,303 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:42,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,304 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,304 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-10-27 23:58:42,304 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:42,304 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-10-27 23:58:42,304 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:42,305 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-10-27 23:58:42,305 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:42,305 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:42,305 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:42,305 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. started... 2023-10-27 23:58:42,305 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1591472e0024853e35a0606e0078390d 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:58:42,729 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228747db2f7a64c4b4e9cd2102ac7af9752 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228747db2f7a64c4b4e9cd2102ac7af9752 2023-10-27 23:58:43,138 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/.tmp/family1/bf2cd723c41d4d8d983de5a11635e9a2 2023-10-27 23:58:43,152 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/.tmp/family1/bf2cd723c41d4d8d983de5a11635e9a2 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/bf2cd723c41d4d8d983de5a11635e9a2 2023-10-27 23:58:43,159 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/bf2cd723c41d4d8d983de5a11635e9a2, entries=1, sequenceid=21, filesize=4.9 K 2023-10-27 23:58:43,160 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1591472e0024853e35a0606e0078390d in 855ms, sequenceid=21, compaction requested=true 2023-10-27 23:58:43,160 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1591472e0024853e35a0606e0078390d: 2023-10-27 23:58:43,161 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:43,161 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:43,161 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:43,161 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:43,161 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-10-27 23:58:43,161 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-10-27 23:58:43,194 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:43,194 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:43,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:43,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:43,195 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,196 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:43,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:43,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:43,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:43,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:43,199 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-10-27 23:58:43,199 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@49f75e41[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:43,199 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-10-27 23:58:43,200 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,210 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,210 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-10-27 23:58:43,210 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-10-27 23:58:43,211 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:43,211 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:43,211 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:43,219 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-10-27 23:58:43,220 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:43,220 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:43,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:43,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:43,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:43,220 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-10-27 23:58:43,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:43,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:43,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:43,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:43,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:43,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:58:43,228 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:58:43,228 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-10-27 23:58:43,228 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:43,228 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:43,228 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,229 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:43,253 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,253 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:43,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:43,253 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:43,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:43,255 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:43,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:43,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:43,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:43,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:43,259 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:43,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:43,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:43,269 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:43,269 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:43,269 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:43,269 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-10-27 23:58:43,269 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:43,269 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:43,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:43,269 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:43,269 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,270 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:43,270 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:43,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:43,270 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:43,270 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:43,271 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,271 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,371 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:43,372 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:43,377 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:43,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-10-27 23:58:43,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,379 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-10-27 23:58:43,379 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:43,380 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-10-27 23:58:43,380 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-10-27 23:58:43,380 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,380 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,386 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:43,386 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:43,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:43,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:43,386 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:43,386 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:43,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,387 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:43,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,387 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,387 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-10-27 23:58:43,388 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:43,388 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-10-27 23:58:43,388 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:43,388 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-10-27 23:58:43,388 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:43,388 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:43,388 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:43,388 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. started... 2023-10-27 23:58:43,388 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1591472e0024853e35a0606e0078390d 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:58:43,815 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512317801588468484cd08cda68874eb31e9a to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512317801588468484cd08cda68874eb31e9a 2023-10-27 23:58:44,222 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/.tmp/family1/6e7a42d059e54398b29e628bc0597e5f 2023-10-27 23:58:44,233 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/.tmp/family1/6e7a42d059e54398b29e628bc0597e5f as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/6e7a42d059e54398b29e628bc0597e5f 2023-10-27 23:58:44,239 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/6e7a42d059e54398b29e628bc0597e5f, entries=1, sequenceid=25, filesize=4.9 K 2023-10-27 23:58:44,240 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1591472e0024853e35a0606e0078390d in 852ms, sequenceid=25, compaction requested=true 2023-10-27 23:58:44,241 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1591472e0024853e35a0606e0078390d: 2023-10-27 23:58:44,241 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:44,241 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:44,241 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:44,241 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:44,241 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-10-27 23:58:44,241 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-10-27 23:58:44,269 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,269 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:44,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:44,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:44,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:44,270 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,270 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:44,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:44,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:44,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:44,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:44,274 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-10-27 23:58:44,274 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@389838cd[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:44,274 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-10-27 23:58:44,274 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,285 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,285 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-10-27 23:58:44,285 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-10-27 23:58:44,285 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:44,285 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:44,285 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:44,294 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-10-27 23:58:44,294 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:44,294 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:44,295 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-10-27 23:58:44,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:44,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:44,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:44,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:44,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:44,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:44,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:44,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:44,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:58:44,304 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:58:44,304 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-10-27 23:58:44,304 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:44,305 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:44,305 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,305 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:44,311 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,311 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:44,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,311 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:44,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:44,312 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:44,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:44,312 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:44,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:44,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:44,314 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:44,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:44,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:44,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:44,326 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:44,326 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:44,326 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:44,326 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-10-27 23:58:44,327 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:44,326 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:44,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:44,326 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:44,326 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,327 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:44,328 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:44,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:44,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:44,327 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-10-27 23:58:44,328 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:44,328 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,328 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,328 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,428 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:44,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:44,440 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:44,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-10-27 23:58:44,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,443 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-10-27 23:58:44,443 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:44,444 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-10-27 23:58:44,444 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-10-27 23:58:44,444 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,444 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,452 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:44,452 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:44,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:44,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:44,453 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:44,453 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:44,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,453 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:44,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,454 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,454 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-10-27 23:58:44,454 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:44,454 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-10-27 23:58:44,454 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:44,455 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-10-27 23:58:44,455 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:44,455 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:44,455 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:44,455 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. started... 2023-10-27 23:58:44,455 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1591472e0024853e35a0606e0078390d 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:58:44,876 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010115bf3375aa0441269b82c009fd576c20 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010115bf3375aa0441269b82c009fd576c20 2023-10-27 23:58:45,288 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/.tmp/family1/5ea0edfaa711462ea76c481a140b8c93 2023-10-27 23:58:45,297 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/.tmp/family1/5ea0edfaa711462ea76c481a140b8c93 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/5ea0edfaa711462ea76c481a140b8c93 2023-10-27 23:58:45,306 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/5ea0edfaa711462ea76c481a140b8c93, entries=1, sequenceid=29, filesize=4.9 K 2023-10-27 23:58:45,307 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1591472e0024853e35a0606e0078390d in 852ms, sequenceid=29, compaction requested=true 2023-10-27 23:58:45,307 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1591472e0024853e35a0606e0078390d: 2023-10-27 23:58:45,308 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:45,308 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:45,308 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:45,308 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:45,308 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-10-27 23:58:45,308 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-10-27 23:58:45,336 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,338 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:45,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:45,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:45,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:45,340 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,340 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:45,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:45,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:45,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:45,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:45,351 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-10-27 23:58:45,351 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4fbb00f4[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:45,351 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-10-27 23:58:45,351 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,361 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,361 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-10-27 23:58:45,361 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-10-27 23:58:45,361 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:45,372 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-10-27 23:58:45,373 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:45,373 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-10-27 23:58:45,375 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:45,376 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:58:45,376 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:45,376 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-10-27 23:58:45,376 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:45,376 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:45,376 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,376 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:45,387 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,387 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:45,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:45,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:45,388 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:45,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:45,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:45,400 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:45,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:45,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,417 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:45,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:45,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:45,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:45,443 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:45,443 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-10-27 23:58:45,444 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:45,445 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,445 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,445 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:45,445 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,445 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,445 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,444 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:45,443 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-10-27 23:58:45,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:45,445 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:45,445 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:45,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:45,446 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:45,446 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:45,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:45,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:45,547 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:45,553 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:45,564 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:45,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-10-27 23:58:45,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,577 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-10-27 23:58:45,577 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:45,585 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-10-27 23:58:45,585 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-10-27 23:58:45,586 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,586 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,595 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:45,596 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:45,596 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:45,596 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:45,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:45,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:45,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,597 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:45,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,598 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,598 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-10-27 23:58:45,598 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:45,598 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-10-27 23:58:45,598 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:45,599 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-10-27 23:58:45,600 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:45,600 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:45,601 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:45,601 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. started... 2023-10-27 23:58:45,601 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1591472e0024853e35a0606e0078390d 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:58:46,043 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601039e341838d3d24475994cb54682b87251 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601039e341838d3d24475994cb54682b87251 2023-10-27 23:58:46,063 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/.tmp/family1/6187726e02254e71af2a1f3d77e61367 2023-10-27 23:58:46,076 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/.tmp/family1/6187726e02254e71af2a1f3d77e61367 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/6187726e02254e71af2a1f3d77e61367 2023-10-27 23:58:46,086 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/6187726e02254e71af2a1f3d77e61367, entries=1, sequenceid=33, filesize=4.9 K 2023-10-27 23:58:46,088 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1591472e0024853e35a0606e0078390d in 487ms, sequenceid=33, compaction requested=true 2023-10-27 23:58:46,088 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1591472e0024853e35a0606e0078390d: 2023-10-27 23:58:46,090 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:46,090 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:46,090 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:46,090 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:46,090 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-10-27 23:58:46,090 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-10-27 23:58:46,119 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:46,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:46,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:46,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:46,119 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:46,120 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,120 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:46,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:46,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:46,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:46,126 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-10-27 23:58:46,126 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@871f0eb[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:46,126 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-10-27 23:58:46,126 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,154 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,154 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-10-27 23:58:46,154 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-10-27 23:58:46,154 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:46,154 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:46,155 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:46,168 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:46,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:46,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:46,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:46,168 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-10-27 23:58:46,168 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:46,168 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-10-27 23:58:46,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:46,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:46,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:46,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:46,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:46,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:58:46,182 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:58:46,182 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-10-27 23:58:46,182 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:46,183 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:46,183 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,183 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:46,194 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,194 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,194 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:46,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:46,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:46,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:46,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:46,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:46,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:46,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,198 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:46,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:46,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:46,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:46,200 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:46,211 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:46,211 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:46,211 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:46,211 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:46,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:46,211 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,211 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:46,211 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,211 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,212 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:46,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:46,212 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:46,212 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-10-27 23:58:46,213 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:46,213 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:46,213 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:46,313 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:46,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:46,320 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:46,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-10-27 23:58:46,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,326 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-10-27 23:58:46,326 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:46,326 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-10-27 23:58:46,326 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-10-27 23:58:46,327 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,327 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,336 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:46,336 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:46,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:46,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:46,338 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:46,338 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:46,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,339 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:46,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,339 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,340 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-10-27 23:58:46,340 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:46,340 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-10-27 23:58:46,340 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:46,341 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-10-27 23:58:46,342 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:46,342 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:46,342 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:46,342 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. started... 2023-10-27 23:58:46,342 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1591472e0024853e35a0606e0078390d 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:58:46,762 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509073ea39371aaf741ab84165e5df67ab540 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201509073ea39371aaf741ab84165e5df67ab540 2023-10-27 23:58:47,178 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/.tmp/family1/2f7464c103784243ae248907cf266a5d 2023-10-27 23:58:47,184 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/.tmp/family1/2f7464c103784243ae248907cf266a5d as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/2f7464c103784243ae248907cf266a5d 2023-10-27 23:58:47,192 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/2f7464c103784243ae248907cf266a5d, entries=1, sequenceid=37, filesize=4.9 K 2023-10-27 23:58:47,195 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1591472e0024853e35a0606e0078390d in 853ms, sequenceid=37, compaction requested=true 2023-10-27 23:58:47,195 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1591472e0024853e35a0606e0078390d: 2023-10-27 23:58:47,195 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:47,195 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:47,195 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:47,195 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:47,195 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-10-27 23:58:47,195 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-10-27 23:58:47,227 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,227 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:47,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:47,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:47,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:47,228 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,228 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:47,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:47,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:47,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:47,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:47,230 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-10-27 23:58:47,230 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b87b2f2[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:47,230 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-10-27 23:58:47,230 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,236 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,236 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-10-27 23:58:47,236 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-10-27 23:58:47,236 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:47,244 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-10-27 23:58:47,244 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:47,244 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-10-27 23:58:47,244 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:47,245 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:58:47,245 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:47,245 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-10-27 23:58:47,245 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:47,245 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:47,245 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,245 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:47,253 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,254 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:47,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,254 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:47,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:47,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:47,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:47,254 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:47,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:47,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:47,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:47,256 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:47,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:47,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:47,261 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:47,261 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,261 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:47,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:47,261 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,261 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:47,261 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-10-27 23:58:47,261 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:47,261 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:47,261 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,261 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-10-27 23:58:47,262 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:47,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:47,261 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,262 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:47,262 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,262 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:47,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:47,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:47,362 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:47,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:47,369 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:47,372 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-10-27 23:58:47,372 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,372 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-10-27 23:58:47,372 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:47,374 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-10-27 23:58:47,374 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-10-27 23:58:47,376 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,376 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,386 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:47,386 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:47,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:47,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:47,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,387 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:47,387 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:47,387 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:47,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,388 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,390 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-10-27 23:58:47,390 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,392 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-10-27 23:58:47,392 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:47,394 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-10-27 23:58:47,394 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:47,394 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:47,394 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:47,394 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. started... 2023-10-27 23:58:47,395 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1591472e0024853e35a0606e0078390d 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:58:47,422 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511203f089713d75343ef9aa13e0cbb259578 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511203f089713d75343ef9aa13e0cbb259578 2023-10-27 23:58:47,439 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/.tmp/family1/b53f730339164117acea92f554e0ffde 2023-10-27 23:58:47,445 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/.tmp/family1/b53f730339164117acea92f554e0ffde as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/b53f730339164117acea92f554e0ffde 2023-10-27 23:58:47,454 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/b53f730339164117acea92f554e0ffde, entries=1, sequenceid=41, filesize=4.9 K 2023-10-27 23:58:47,455 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1591472e0024853e35a0606e0078390d in 61ms, sequenceid=41, compaction requested=true 2023-10-27 23:58:47,455 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1591472e0024853e35a0606e0078390d: 2023-10-27 23:58:47,455 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:47,455 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:47,455 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:47,456 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:47,456 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-10-27 23:58:47,456 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-10-27 23:58:47,461 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:47,461 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:47,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:47,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:47,461 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:47,461 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:47,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:47,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:47,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:47,465 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-10-27 23:58:47,466 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@220b31f[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:47,466 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-10-27 23:58:47,466 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,469 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,470 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-10-27 23:58:47,470 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-10-27 23:58:47,470 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:47,478 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-10-27 23:58:47,478 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:47,478 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-10-27 23:58:47,479 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:47,485 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:58:47,485 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:47,485 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-10-27 23:58:47,485 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:47,485 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:47,485 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,485 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:47,502 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:47,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:47,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:47,503 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,503 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:47,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,504 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:47,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:47,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:47,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:47,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:47,505 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:47,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:47,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:47,511 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:47,511 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:47,511 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:47,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:47,511 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:47,511 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,511 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-10-27 23:58:47,512 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:47,511 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-10-27 23:58:47,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:47,511 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,512 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:47,512 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,512 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,513 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,513 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:47,513 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:47,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:47,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:47,613 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:47,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:47,617 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36413] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-10-27 23:58:47,618 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] procedure2.ProcedureExecutor(1028): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-10-27 23:58:47,619 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-10-27 23:58:47,622 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-27 23:58:47,622 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698451727622 2023-10-27 23:58:47,625 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-10-27 23:58:47,626 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-10-27 23:58:47,626 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-10-27 23:58:47,628 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] zookeeper.ReadOnlyZKClient(139): Connect 0x5073ebf1 to 127.0.0.1:57369 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-10-27 23:58:47,645 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@454f2813, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-27 23:58:47,645 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-10-27 23:58:48,120 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010348240c763c20420182d3609c91257392 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010348240c763c20420182d3609c91257392 2023-10-27 23:58:48,122 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-10-27 23:58:48,140 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-10-27 23:58:48,141 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:37876, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-10-27 23:58:48,156 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-10-27 23:58:48,157 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:32932, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-10-27 23:58:48,179 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/ee609f2a7dca442181803a5bd342076a first=Optional[r5] last=Optional[r6] 2023-10-27 23:58:48,182 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/ee609f2a7dca442181803a5bd342076a}] 2023-10-27 23:58:48,189 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/ee609f2a7dca442181803a5bd342076a for inclusion in 1591472e0024853e35a0606e0078390d/family1 2023-10-27 23:58:48,193 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-10-27 23:58:48,193 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(777): Region bounds: first= last= 2023-10-27 23:58:48,193 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for 1591472e0024853e35a0606e0078390d: 2023-10-27 23:58:48,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/ee609f2a7dca442181803a5bd342076a to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithMonthlyPolicy__k6cr9g1fkb72uf70u8q70mt4e0lv6kk66u4ab8p2fag832224v5obrsl1gjbq42o/family1/ee609f2a7dca442181803a5bd342076a 2023-10-27 23:58:48,197 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithMonthlyPolicy__k6cr9g1fkb72uf70u8q70mt4e0lv6kk66u4ab8p2fag832224v5obrsl1gjbq42o/family1/ee609f2a7dca442181803a5bd342076a as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/21326a0a679f4be2975f6691f7522596_SeqId_44_ 2023-10-27 23:58:48,198 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithMonthlyPolicy__k6cr9g1fkb72uf70u8q70mt4e0lv6kk66u4ab8p2fag832224v5obrsl1gjbq42o/family1/ee609f2a7dca442181803a5bd342076a into 1591472e0024853e35a0606e0078390d/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/21326a0a679f4be2975f6691f7522596_SeqId_44_ - updating store file list. 2023-10-27 23:58:48,214 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/21326a0a679f4be2975f6691f7522596_SeqId_44_ into 1591472e0024853e35a0606e0078390d/family1 2023-10-27 23:58:48,214 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithMonthlyPolicy__k6cr9g1fkb72uf70u8q70mt4e0lv6kk66u4ab8p2fag832224v5obrsl1gjbq42o/family1/ee609f2a7dca442181803a5bd342076a into 1591472e0024853e35a0606e0078390d/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/21326a0a679f4be2975f6691f7522596_SeqId_44_) 2023-10-27 23:58:48,215 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithMonthlyPolicy__k6cr9g1fkb72uf70u8q70mt4e0lv6kk66u4ab8p2fag832224v5obrsl1gjbq42o/family1/ee609f2a7dca442181803a5bd342076a 2023-10-27 23:58:48,215 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-10-27 23:58:48,225 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-10-27 23:58:48,225 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(1912): 1591472e0024853e35a0606e0078390d/family1 is initiating minor compaction 2023-10-27 23:58:48,225 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@248ee131; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-27 23:58:48,225 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 1591472e0024853e35a0606e0078390d/family1 in testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:48,226 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/94d41279f5de4ddda3c43d64f11d7b20, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/75a0dbe8cf6e4e50a74f7ac89eda1f55, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/faa02e6bf99f4ba7a9fbc153f8f4dce9, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/bf2cd723c41d4d8d983de5a11635e9a2, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/6e7a42d059e54398b29e628bc0597e5f, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/5ea0edfaa711462ea76c481a140b8c93, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/6187726e02254e71af2a1f3d77e61367, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/2f7464c103784243ae248907cf266a5d, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/b53f730339164117acea92f554e0ffde, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/21326a0a679f4be2975f6691f7522596_SeqId_44_] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/.tmp, totalSize=49.7 K 2023-10-27 23:58:48,226 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 94d41279f5de4ddda3c43d64f11d7b20, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-10-27 23:58:48,226 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 75a0dbe8cf6e4e50a74f7ac89eda1f55, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-10-27 23:58:48,226 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting faa02e6bf99f4ba7a9fbc153f8f4dce9, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-10-27 23:58:48,226 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting bf2cd723c41d4d8d983de5a11635e9a2, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-10-27 23:58:48,226 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 6e7a42d059e54398b29e628bc0597e5f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-10-27 23:58:48,226 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 5ea0edfaa711462ea76c481a140b8c93, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-10-27 23:58:48,227 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 6187726e02254e71af2a1f3d77e61367, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-10-27 23:58:48,227 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 2f7464c103784243ae248907cf266a5d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-10-27 23:58:48,227 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting b53f730339164117acea92f554e0ffde, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-10-27 23:58:48,227 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 21326a0a679f4be2975f6691f7522596_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-10-27 23:58:48,232 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-10-27 23:58:48,236 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:58:48,240 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010115bf3375aa0441269b82c009fd576c20 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010115bf3375aa0441269b82c009fd576c20 2023-10-27 23:58:48,242 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601039e341838d3d24475994cb54682b87251 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601039e341838d3d24475994cb54682b87251 2023-10-27 23:58:48,242 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-10-27 23:58:48,242 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-10-27 23:58:48,242 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-10-27 23:58:48,242 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-10-27 23:58:48,309 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1591472e0024853e35a0606e0078390d#family1#compaction#22 average throughput is 0.23 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-10-27 23:58:49,119 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231a677dc41318748b69f4a1ee11d5bbcb8 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231a677dc41318748b69f4a1ee11d5bbcb8 2023-10-27 23:58:49,121 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-10-27 23:58:49,140 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/.tmp/family1/7933bd085fe543e0be416891ffccbe1b as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/7933bd085fe543e0be416891ffccbe1b 2023-10-27 23:58:49,150 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 1591472e0024853e35a0606e0078390d/family1 of 1591472e0024853e35a0606e0078390d into 7933bd085fe543e0be416891ffccbe1b(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-10-27 23:58:49,150 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1591472e0024853e35a0606e0078390d: 2023-10-27 23:58:49,150 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1698451128225; duration=0sec 2023-10-27 23:58:49,150 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-27 23:58:49,155 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/87be53008b834152863704ab56c141b1 first=Optional[r1] last=Optional[r4] 2023-10-27 23:58:49,157 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/87be53008b834152863704ab56c141b1}] 2023-10-27 23:58:49,161 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/87be53008b834152863704ab56c141b1 for inclusion in 1591472e0024853e35a0606e0078390d/family1 2023-10-27 23:58:49,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-10-27 23:58:49,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(777): Region bounds: first= last= 2023-10-27 23:58:49,168 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for 1591472e0024853e35a0606e0078390d: 2023-10-27 23:58:49,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/87be53008b834152863704ab56c141b1 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithMonthlyPolicy__9r9437cro83u28edt06l2p5v427vra39fhsjc1sejmnsebjud4altd7q8cqolg3g/family1/87be53008b834152863704ab56c141b1 2023-10-27 23:58:49,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithMonthlyPolicy__9r9437cro83u28edt06l2p5v427vra39fhsjc1sejmnsebjud4altd7q8cqolg3g/family1/87be53008b834152863704ab56c141b1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/adf4773f1dfe4ae59cd02763ad12856e_SeqId_47_ 2023-10-27 23:58:49,185 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithMonthlyPolicy__9r9437cro83u28edt06l2p5v427vra39fhsjc1sejmnsebjud4altd7q8cqolg3g/family1/87be53008b834152863704ab56c141b1 into 1591472e0024853e35a0606e0078390d/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/adf4773f1dfe4ae59cd02763ad12856e_SeqId_47_ - updating store file list. 2023-10-27 23:58:49,190 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/adf4773f1dfe4ae59cd02763ad12856e_SeqId_47_ into 1591472e0024853e35a0606e0078390d/family1 2023-10-27 23:58:49,190 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithMonthlyPolicy__9r9437cro83u28edt06l2p5v427vra39fhsjc1sejmnsebjud4altd7q8cqolg3g/family1/87be53008b834152863704ab56c141b1 into 1591472e0024853e35a0606e0078390d/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/adf4773f1dfe4ae59cd02763ad12856e_SeqId_47_) 2023-10-27 23:58:49,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithMonthlyPolicy__9r9437cro83u28edt06l2p5v427vra39fhsjc1sejmnsebjud4altd7q8cqolg3g/family1/87be53008b834152863704ab56c141b1 2023-10-27 23:58:49,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-10-27 23:58:49,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-10-27 23:58:49,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(1912): 1591472e0024853e35a0606e0078390d/family1 is initiating minor compaction (all files) 2023-10-27 23:58:49,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@464cd0c8; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-27 23:58:49,195 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 1591472e0024853e35a0606e0078390d/family1 in testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-27 23:58:49,196 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/4e2dad885bee4440babf05a73e3a80d6, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/7933bd085fe543e0be416891ffccbe1b, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/adf4773f1dfe4ae59cd02763ad12856e_SeqId_47_] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/.tmp, totalSize=178.1 K 2023-10-27 23:58:49,196 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 4e2dad885bee4440babf05a73e3a80d6, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878821089 2023-10-27 23:58:49,197 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 7933bd085fe543e0be416891ffccbe1b, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621221089 2023-10-27 23:58:49,197 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting adf4773f1dfe4ae59cd02763ad12856e_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965221089 2023-10-27 23:58:49,197 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-10-27 23:58:49,198 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:58:49,200 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201f53d6c451537482b96f1bd03760f6dad to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201f53d6c451537482b96f1bd03760f6dad 2023-10-27 23:58:49,201 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205877f4a2801bc4241a7701c02e0680943 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205877f4a2801bc4241a7701c02e0680943 2023-10-27 23:58:49,202 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228747db2f7a64c4b4e9cd2102ac7af9752 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228747db2f7a64c4b4e9cd2102ac7af9752 2023-10-27 23:58:49,204 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512317801588468484cd08cda68874eb31e9a to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512317801588468484cd08cda68874eb31e9a 2023-10-27 23:58:49,204 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-10-27 23:58:49,204 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-10-27 23:58:49,205 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-10-27 23:58:49,205 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-10-27 23:58:49,217 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1591472e0024853e35a0606e0078390d#family1#compaction#23 average throughput is 15.56 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-10-27 23:58:49,255 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/94d41279f5de4ddda3c43d64f11d7b20, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/75a0dbe8cf6e4e50a74f7ac89eda1f55, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/faa02e6bf99f4ba7a9fbc153f8f4dce9, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/bf2cd723c41d4d8d983de5a11635e9a2, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/6e7a42d059e54398b29e628bc0597e5f, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/5ea0edfaa711462ea76c481a140b8c93, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/6187726e02254e71af2a1f3d77e61367, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/2f7464c103784243ae248907cf266a5d, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/b53f730339164117acea92f554e0ffde, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/21326a0a679f4be2975f6691f7522596_SeqId_44_] to archive 2023-10-27 23:58:49,256 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:58:49,269 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/94d41279f5de4ddda3c43d64f11d7b20 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/94d41279f5de4ddda3c43d64f11d7b20 2023-10-27 23:58:49,289 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/75a0dbe8cf6e4e50a74f7ac89eda1f55 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/75a0dbe8cf6e4e50a74f7ac89eda1f55 2023-10-27 23:58:49,296 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/faa02e6bf99f4ba7a9fbc153f8f4dce9 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/faa02e6bf99f4ba7a9fbc153f8f4dce9 2023-10-27 23:58:49,298 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/bf2cd723c41d4d8d983de5a11635e9a2 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/bf2cd723c41d4d8d983de5a11635e9a2 2023-10-27 23:58:49,309 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/6e7a42d059e54398b29e628bc0597e5f to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/6e7a42d059e54398b29e628bc0597e5f 2023-10-27 23:58:49,315 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/5ea0edfaa711462ea76c481a140b8c93 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/5ea0edfaa711462ea76c481a140b8c93 2023-10-27 23:58:49,317 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/6187726e02254e71af2a1f3d77e61367 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/6187726e02254e71af2a1f3d77e61367 2023-10-27 23:58:49,319 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/2f7464c103784243ae248907cf266a5d to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/2f7464c103784243ae248907cf266a5d 2023-10-27 23:58:49,320 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/b53f730339164117acea92f554e0ffde to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/b53f730339164117acea92f554e0ffde 2023-10-27 23:58:49,322 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/21326a0a679f4be2975f6691f7522596_SeqId_44_ to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/21326a0a679f4be2975f6691f7522596_SeqId_44_ 2023-10-27 23:58:49,723 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130772d1164cc154102a9f0eef14ca2f7a0 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130772d1164cc154102a9f0eef14ca2f7a0 2023-10-27 23:58:49,724 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-10-27 23:58:49,763 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/86c2557737c94ca39dfa6997cd1ea13a first=Optional[r00] last=Optional[r8] 2023-10-27 23:58:49,764 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/86c2557737c94ca39dfa6997cd1ea13a}] 2023-10-27 23:58:49,766 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/86c2557737c94ca39dfa6997cd1ea13a for inclusion in 1591472e0024853e35a0606e0078390d/family1 2023-10-27 23:58:49,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-10-27 23:58:49,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(777): Region bounds: first= last= 2023-10-27 23:58:49,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for 1591472e0024853e35a0606e0078390d: 2023-10-27 23:58:49,775 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/86c2557737c94ca39dfa6997cd1ea13a to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithMonthlyPolicy__gbk6ellgv3tjee5ukefvbaq76s3g01ahc5nskh85mete793r1qij8bv8cb8i710q/family1/86c2557737c94ca39dfa6997cd1ea13a 2023-10-27 23:58:49,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithMonthlyPolicy__gbk6ellgv3tjee5ukefvbaq76s3g01ahc5nskh85mete793r1qij8bv8cb8i710q/family1/86c2557737c94ca39dfa6997cd1ea13a as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/627a8663394547a3b4a304c51d38b2d7_SeqId_49_ 2023-10-27 23:58:49,779 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithMonthlyPolicy__gbk6ellgv3tjee5ukefvbaq76s3g01ahc5nskh85mete793r1qij8bv8cb8i710q/family1/86c2557737c94ca39dfa6997cd1ea13a into 1591472e0024853e35a0606e0078390d/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/627a8663394547a3b4a304c51d38b2d7_SeqId_49_ - updating store file list. 2023-10-27 23:58:49,786 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/627a8663394547a3b4a304c51d38b2d7_SeqId_49_ into 1591472e0024853e35a0606e0078390d/family1 2023-10-27 23:58:49,786 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithMonthlyPolicy__gbk6ellgv3tjee5ukefvbaq76s3g01ahc5nskh85mete793r1qij8bv8cb8i710q/family1/86c2557737c94ca39dfa6997cd1ea13a into 1591472e0024853e35a0606e0078390d/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/627a8663394547a3b4a304c51d38b2d7_SeqId_49_) 2023-10-27 23:58:49,787 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithMonthlyPolicy__gbk6ellgv3tjee5ukefvbaq76s3g01ahc5nskh85mete793r1qij8bv8cb8i710q/family1/86c2557737c94ca39dfa6997cd1ea13a 2023-10-27 23:58:49,789 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-10-27 23:58:49,789 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:58:49,791 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511203f089713d75343ef9aa13e0cbb259578 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511203f089713d75343ef9aa13e0cbb259578 2023-10-27 23:58:49,792 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112896a9a7bd579f4c0c8977379aa24666ce to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112896a9a7bd579f4c0c8977379aa24666ce 2023-10-27 23:58:49,793 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511309c4b73e28ec045b49348401231ab00dd to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511309c4b73e28ec045b49348401231ab00dd 2023-10-27 23:58:49,793 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-10-27 23:58:49,793 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-10-27 23:58:49,793 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-10-27 23:58:49,793 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-10-27 23:58:49,793 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-10-27 23:58:49,793 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-10-27 23:58:49,794 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-27 23:58:49,795 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-10-27 23:58:49,796 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 2.1780 sec 2023-10-27 23:58:50,032 INFO [Listener at localhost/40295] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=265 (was 249) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:59338 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-217416042-188.40.62.62-1698451092001:blk_1073741898_1074, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:59220 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-747715023_17 at /127.0.0.1:59398 [Receiving block BP-217416042-188.40.62.62-1698451092001:blk_1073741898_1074] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x52087bb3-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:59290 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-747715023_17 at /127.0.0.1:59358 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57369@0x5073ebf1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$46/154844478.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:59416 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/cluster_1ba21e36-f5ca-27f1-0277-ca251e3b747a/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6e7da2c7-shared-pool-23 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231789134f4f2bf43a2b119ccd8312cc567 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: hconnection-0x52087bb3-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:59332 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:59166 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-747715023_17 at /127.0.0.1:59200 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:59368 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:59212 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:59304 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x52087bb3-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:59262 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:59224 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x52087bb3-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:59260 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:59424 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x52087bb3-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:59342 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ResponseProcessor for block BP-217416042-188.40.62.62-1698451092001:blk_1073741898_1074 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:118) java.io.FilterInputStream.read(FilterInputStream.java:83) java.io.FilterInputStream.read(FilterInputStream.java:83) org.apache.hadoop.hdfs.protocolPB.PBHelperClient.vintPrefixed(PBHelperClient.java:454) org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:213) org.apache.hadoop.hdfs.DataStreamer$ResponseProcessor.run(DataStreamer.java:1080) Potentially hanging thread: hconnection-0x52087bb3-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/.tmp/family1/87da79e62c5a48e6a4a756345febe574 block BP-217416042-188.40.62.62-1698451092001:blk_1073741898_1074 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:59138 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57369@0x5073ebf1-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:59276 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57369@0x5073ebf1-SendThread(127.0.0.1:57369) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:59344 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x52087bb3-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_694505983_17 at /127.0.0.1:59320 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=632 (was 591) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=250 (was 183) - SystemLoadAverage LEAK? -, ProcessCount=167 (was 167), AvailableMemoryMB=4973 (was 5029) 2023-10-27 23:58:50,053 INFO [Listener at localhost/40295] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=265, OpenFileDescriptor=632, MaxFileDescriptor=60000, SystemLoadAverage=250, ProcessCount=167, AvailableMemoryMB=4972 2023-10-27 23:58:50,056 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.HMaster$4(2112): Client=jenkins//188.40.62.62 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-10-27 23:58:50,058 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure2.ProcedureExecutor(1028): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:50,060 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-10-27 23:58:50,061 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(700): Client=jenkins//188.40.62.62 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-10-27 23:58:50,062 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-10-27 23:58:50,062 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-10-27 23:58:50,066 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c 2023-10-27 23:58:50,068 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c empty. 2023-10-27 23:58:50,068 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c 2023-10-27 23:58:50,068 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-10-27 23:58:50,108 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/.tmp/family1/87da79e62c5a48e6a4a756345febe574 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/87da79e62c5a48e6a4a756345febe574 2023-10-27 23:58:50,115 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 1591472e0024853e35a0606e0078390d/family1 of 1591472e0024853e35a0606e0078390d into 87da79e62c5a48e6a4a756345febe574(size=167.4 K), total size for store is 333.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-10-27 23:58:50,115 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1591472e0024853e35a0606e0078390d: 2023-10-27 23:58:50,115 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1698451129195; duration=0sec 2023-10-27 23:58:50,115 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-27 23:58:50,129 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/4e2dad885bee4440babf05a73e3a80d6, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/7933bd085fe543e0be416891ffccbe1b, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/adf4773f1dfe4ae59cd02763ad12856e_SeqId_47_] to archive 2023-10-27 23:58:50,132 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:58:50,135 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/4e2dad885bee4440babf05a73e3a80d6 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/4e2dad885bee4440babf05a73e3a80d6 2023-10-27 23:58:50,139 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/7933bd085fe543e0be416891ffccbe1b to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/7933bd085fe543e0be416891ffccbe1b 2023-10-27 23:58:50,142 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/adf4773f1dfe4ae59cd02763ad12856e_SeqId_47_ to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/family1/adf4773f1dfe4ae59cd02763ad12856e_SeqId_47_ 2023-10-27 23:58:50,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-10-27 23:58:50,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-10-27 23:58:50,484 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-10-27 23:58:50,485 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => f7ae7da0566a658f8f0ae5d6c67c438c, NAME => 'testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp 2023-10-27 23:58:50,496 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:58:50,496 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing f7ae7da0566a658f8f0ae5d6c67c438c, disabling compactions & flushes 2023-10-27 23:58:50,496 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:58:50,496 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:58:50,496 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. after waiting 0 ms 2023-10-27 23:58:50,497 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:58:50,497 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:58:50,497 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for f7ae7da0566a658f8f0ae5d6c67c438c: 2023-10-27 23:58:50,500 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-10-27 23:58:50,503 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1698451130503"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451130503"}]},"ts":"1698451130503"} 2023-10-27 23:58:50,505 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-10-27 23:58:50,506 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-10-27 23:58:50,506 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451130506"}]},"ts":"1698451130506"} 2023-10-27 23:58:50,508 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-10-27 23:58:50,580 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=f7ae7da0566a658f8f0ae5d6c67c438c, ASSIGN}] 2023-10-27 23:58:50,583 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=f7ae7da0566a658f8f0ae5d6c67c438c, ASSIGN 2023-10-27 23:58:50,585 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=f7ae7da0566a658f8f0ae5d6c67c438c, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,43469,1698451096272; forceNewPlan=false, retain=false 2023-10-27 23:58:50,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-10-27 23:58:50,736 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=f7ae7da0566a658f8f0ae5d6c67c438c, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:50,736 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1698451130736"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451130736"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451130736"}]},"ts":"1698451130736"} 2023-10-27 23:58:50,738 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure f7ae7da0566a658f8f0ae5d6c67c438c, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-27 23:58:50,892 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:50,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:58:50,897 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f7ae7da0566a658f8f0ae5d6c67c438c, NAME => 'testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c.', STARTKEY => '', ENDKEY => ''} 2023-10-27 23:58:50,897 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy f7ae7da0566a658f8f0ae5d6c67c438c 2023-10-27 23:58:50,897 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:58:50,897 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for f7ae7da0566a658f8f0ae5d6c67c438c 2023-10-27 23:58:50,897 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for f7ae7da0566a658f8f0ae5d6c67c438c 2023-10-27 23:58:50,899 INFO [StoreOpener-f7ae7da0566a658f8f0ae5d6c67c438c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f7ae7da0566a658f8f0ae5d6c67c438c 2023-10-27 23:58:50,900 DEBUG [StoreOpener-f7ae7da0566a658f8f0ae5d6c67c438c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1 2023-10-27 23:58:50,900 DEBUG [StoreOpener-f7ae7da0566a658f8f0ae5d6c67c438c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1 2023-10-27 23:58:50,901 INFO [StoreOpener-f7ae7da0566a658f8f0ae5d6c67c438c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f7ae7da0566a658f8f0ae5d6c67c438c columnFamilyName family1 2023-10-27 23:58:50,902 INFO [StoreOpener-f7ae7da0566a658f8f0ae5d6c67c438c-1] regionserver.HStore(310): Store=f7ae7da0566a658f8f0ae5d6c67c438c/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-27 23:58:50,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c 2023-10-27 23:58:50,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c 2023-10-27 23:58:50,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for f7ae7da0566a658f8f0ae5d6c67c438c 2023-10-27 23:58:50,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-10-27 23:58:50,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened f7ae7da0566a658f8f0ae5d6c67c438c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64558422, jitterRate=-0.038004547357559204}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-10-27 23:58:50,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for f7ae7da0566a658f8f0ae5d6c67c438c: 2023-10-27 23:58:50,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c., pid=25, masterSystemTime=1698451130892 2023-10-27 23:58:50,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:58:50,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:58:50,914 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=f7ae7da0566a658f8f0ae5d6c67c438c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:50,914 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1698451130914"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451130914"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451130914"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451130914"}]},"ts":"1698451130914"} 2023-10-27 23:58:50,918 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=25, resume processing ppid=24 2023-10-27 23:58:50,918 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure f7ae7da0566a658f8f0ae5d6c67c438c, server=jenkins-hbase16.apache.org,43469,1698451096272 in 178 msec 2023-10-27 23:58:50,920 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=24, resume processing ppid=23 2023-10-27 23:58:50,920 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=f7ae7da0566a658f8f0ae5d6c67c438c, ASSIGN in 338 msec 2023-10-27 23:58:50,921 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-10-27 23:58:50,921 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451130921"}]},"ts":"1698451130921"} 2023-10-27 23:58:50,922 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-10-27 23:58:50,938 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-10-27 23:58:50,940 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 882 msec 2023-10-27 23:58:51,168 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-10-27 23:58:51,169 INFO [Listener at localhost/40295] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-10-27 23:58:51,208 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:51,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,212 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-10-27 23:58:51,212 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:51,212 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-10-27 23:58:51,212 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-10-27 23:58:51,212 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,212 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,219 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,219 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:51,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:51,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:51,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,220 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,221 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,221 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:51,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:51,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,222 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,223 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,223 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-10-27 23:58:51,223 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:51,223 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-10-27 23:58:51,225 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:51,225 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:51,225 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:58:51,225 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. started... 2023-10-27 23:58:51,226 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f7ae7da0566a658f8f0ae5d6c67c438c 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-10-27 23:58:51,269 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130368e04b2adbc454db45879ca087335e8 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151130368e04b2adbc454db45879ca087335e8 2023-10-27 23:58:51,675 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/.tmp/family1/1b66eb8acac8414a96612d5ace3103df 2023-10-27 23:58:51,684 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/.tmp/family1/1b66eb8acac8414a96612d5ace3103df as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/1b66eb8acac8414a96612d5ace3103df 2023-10-27 23:58:51,689 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/1b66eb8acac8414a96612d5ace3103df, entries=1000, sequenceid=5, filesize=165.1 K 2023-10-27 23:58:51,692 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for f7ae7da0566a658f8f0ae5d6c67c438c in 466ms, sequenceid=5, compaction requested=false 2023-10-27 23:58:51,692 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-10-27 23:58:51,693 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f7ae7da0566a658f8f0ae5d6c67c438c: 2023-10-27 23:58:51,693 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:58:51,693 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:51,693 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:51,693 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,693 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-10-27 23:58:51,693 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-10-27 23:58:51,727 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,727 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:51,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:51,728 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,728 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:51,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:51,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:51,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:51,729 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-10-27 23:58:51,729 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-10-27 23:58:51,729 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@150bbf0f[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:51,729 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,736 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,736 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,736 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:51,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,736 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-10-27 23:58:51,736 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-10-27 23:58:51,736 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:51,744 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,744 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-10-27 23:58:51,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:51,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:51,744 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:51,744 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-10-27 23:58:51,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:51,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:51,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:51,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:58:51,747 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:58:51,747 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-10-27 23:58:51,747 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:51,747 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:51,747 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,747 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:51,752 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,752 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:51,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:51,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,752 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:51,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:51,753 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:51,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:51,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:51,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,754 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:51,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,761 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,761 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:51,761 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:51,761 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:51,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:51,761 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-10-27 23:58:51,761 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:51,761 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,761 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:51,761 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,761 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,761 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:51,761 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,761 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:51,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:51,761 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:51,862 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:51,862 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:51,868 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:51,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-10-27 23:58:51,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,870 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-10-27 23:58:51,870 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:51,870 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-10-27 23:58:51,870 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-10-27 23:58:51,871 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,871 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,877 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:51,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:51,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:51,877 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,879 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,879 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:51,879 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:51,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,880 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,880 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-10-27 23:58:51,880 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,880 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-10-27 23:58:51,880 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:51,880 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-10-27 23:58:51,881 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:51,881 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:51,881 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:58:51,881 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. started... 2023-10-27 23:58:51,881 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f7ae7da0566a658f8f0ae5d6c67c438c 1/1 column families, dataSize=74 B heapSize=408 B 2023-10-27 23:58:51,900 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511283849d1656e2d4b88955d71a47494b9f8 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201511283849d1656e2d4b88955d71a47494b9f8 2023-10-27 23:58:51,914 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/.tmp/family1/c84ccc42ea274ead85b173f0d69c71d3 2023-10-27 23:58:51,923 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/.tmp/family1/c84ccc42ea274ead85b173f0d69c71d3 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/c84ccc42ea274ead85b173f0d69c71d3 2023-10-27 23:58:51,931 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/c84ccc42ea274ead85b173f0d69c71d3, entries=1, sequenceid=9, filesize=5.0 K 2023-10-27 23:58:51,932 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for f7ae7da0566a658f8f0ae5d6c67c438c in 51ms, sequenceid=9, compaction requested=false 2023-10-27 23:58:51,932 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f7ae7da0566a658f8f0ae5d6c67c438c: 2023-10-27 23:58:51,932 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:58:51,932 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:51,932 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:51,932 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,932 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-10-27 23:58:51,932 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-10-27 23:58:51,943 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:51,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:51,943 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:51,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:51,944 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,944 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:51,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:51,946 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-10-27 23:58:51,946 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@12fae3ed[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:51,946 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-10-27 23:58:51,946 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,952 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,952 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-10-27 23:58:51,952 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-10-27 23:58:51,953 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:51,953 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,953 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:51,961 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-10-27 23:58:51,961 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,961 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:51,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:51,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:51,961 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-10-27 23:58:51,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:51,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:51,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:51,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:58:51,965 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:58:51,965 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-10-27 23:58:51,965 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:51,965 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:51,965 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,965 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:51,976 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,976 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:51,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,976 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:51,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:51,976 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:51,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:51,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:51,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:51,977 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:51,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,986 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,986 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:51,986 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:51,986 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-10-27 23:58:51,986 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:51,986 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:51,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:51,986 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:51,986 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,986 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:51,986 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:51,986 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:51,987 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,987 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,987 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:51,988 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-10-27 23:58:51,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:51,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:52,087 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:52,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:52,092 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:52,095 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-10-27 23:58:52,095 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:52,095 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-10-27 23:58:52,095 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:52,095 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-10-27 23:58:52,095 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-10-27 23:58:52,095 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:52,095 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:52,102 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:52,103 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:52,103 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:52,103 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:52,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:52,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:52,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:52,104 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:52,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:52,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:52,105 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:52,106 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-10-27 23:58:52,106 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:52,106 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-10-27 23:58:52,106 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:52,106 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-10-27 23:58:52,106 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:52,106 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:52,107 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:58:52,107 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. started... 2023-10-27 23:58:52,107 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f7ae7da0566a658f8f0ae5d6c67c438c 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:58:52,530 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201e8c7717728d444aeb7f5741f11c1d65c to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201e8c7717728d444aeb7f5741f11c1d65c 2023-10-27 23:58:52,936 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/.tmp/family1/3920e0e4eeae4d4396c3b00722ac1475 2023-10-27 23:58:52,945 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/.tmp/family1/3920e0e4eeae4d4396c3b00722ac1475 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/3920e0e4eeae4d4396c3b00722ac1475 2023-10-27 23:58:52,960 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/3920e0e4eeae4d4396c3b00722ac1475, entries=1, sequenceid=13, filesize=4.9 K 2023-10-27 23:58:52,961 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f7ae7da0566a658f8f0ae5d6c67c438c in 854ms, sequenceid=13, compaction requested=true 2023-10-27 23:58:52,961 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f7ae7da0566a658f8f0ae5d6c67c438c: 2023-10-27 23:58:52,961 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:58:52,961 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:52,961 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:52,961 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:52,961 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-10-27 23:58:52,961 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-10-27 23:58:52,994 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:52,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:52,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:52,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:52,994 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:52,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:52,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:52,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:52,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:52,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:52,996 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-10-27 23:58:52,996 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@54acd362[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:52,996 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-10-27 23:58:52,996 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:52,997 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:52,997 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:53,003 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:53,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:53,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:53,003 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-10-27 23:58:53,003 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-10-27 23:58:53,003 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:53,007 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:53,007 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:53,011 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:53,011 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-10-27 23:58:53,011 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:53,011 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-10-27 23:58:53,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:53,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:53,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:53,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:53,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:53,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:53,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:53,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:53,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:53,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:53,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:58:53,021 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:58:53,021 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-10-27 23:58:53,021 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:53,021 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:53,021 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:53,021 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:53,034 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:53,034 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:53,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:53,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:53,035 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:53,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:53,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:53,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:53,036 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:53,039 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:53,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:53,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:53,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:53,061 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:53,061 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:53,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:53,061 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:53,061 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:53,061 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-10-27 23:58:53,062 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:53,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:53,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:53,061 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:53,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:53,062 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:53,062 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:53,062 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:53,062 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:53,062 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:53,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:53,062 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:53,062 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:53,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:53,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:53,063 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-10-27 23:58:53,162 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:53,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:53,169 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:53,175 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-10-27 23:58:53,175 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:53,175 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-10-27 23:58:53,175 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:53,175 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-10-27 23:58:53,175 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-10-27 23:58:53,176 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:53,176 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:53,186 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:53,186 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:53,186 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:53,187 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:53,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:53,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:53,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:53,187 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:53,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:53,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:53,188 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:53,190 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-10-27 23:58:53,190 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:53,190 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-10-27 23:58:53,190 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:53,190 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-10-27 23:58:53,190 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:53,190 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:53,190 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:58:53,190 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. started... 2023-10-27 23:58:53,190 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f7ae7da0566a658f8f0ae5d6c67c438c 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:58:53,616 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205411ad5847b5a430dbe5a2d15f7ea4e55 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205411ad5847b5a430dbe5a2d15f7ea4e55 2023-10-27 23:58:54,024 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/.tmp/family1/0655698fba01487f87e9cfa70c206384 2023-10-27 23:58:54,036 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/.tmp/family1/0655698fba01487f87e9cfa70c206384 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/0655698fba01487f87e9cfa70c206384 2023-10-27 23:58:54,041 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/0655698fba01487f87e9cfa70c206384, entries=1, sequenceid=17, filesize=4.9 K 2023-10-27 23:58:54,042 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f7ae7da0566a658f8f0ae5d6c67c438c in 852ms, sequenceid=17, compaction requested=true 2023-10-27 23:58:54,043 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f7ae7da0566a658f8f0ae5d6c67c438c: 2023-10-27 23:58:54,043 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:58:54,043 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:54,043 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:54,043 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:54,043 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-10-27 23:58:54,043 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-10-27 23:58:54,077 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:54,077 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:54,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:54,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:54,078 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,078 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:54,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:54,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:54,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:54,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:54,080 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-10-27 23:58:54,080 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-10-27 23:58:54,080 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53932aa3[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:54,080 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,085 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,086 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-10-27 23:58:54,086 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-10-27 23:58:54,086 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:54,086 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:54,086 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:54,094 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-10-27 23:58:54,094 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:54,094 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:54,094 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-10-27 23:58:54,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:54,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:54,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:54,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:54,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:54,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:54,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:54,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:54,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:58:54,097 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:58:54,097 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-10-27 23:58:54,097 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:54,097 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:54,097 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,097 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:54,102 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,102 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:54,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:54,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,102 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:54,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:54,103 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:54,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:54,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:54,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:54,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,104 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:54,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:54,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:54,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:54,110 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:54,110 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:54,111 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-10-27 23:58:54,111 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:54,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:54,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:54,111 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:54,111 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-10-27 23:58:54,111 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:54,111 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,111 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:54,111 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:54,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:54,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:54,111 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,112 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:54,112 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,112 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,112 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,211 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:54,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:54,216 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:54,217 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-10-27 23:58:54,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,218 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-10-27 23:58:54,218 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:54,218 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-10-27 23:58:54,218 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-10-27 23:58:54,218 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,218 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,227 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:54,227 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:54,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:54,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:54,228 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:54,228 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:54,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,228 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:54,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,229 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,229 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-10-27 23:58:54,229 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:54,229 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-10-27 23:58:54,229 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:54,229 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-10-27 23:58:54,229 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:54,229 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:54,229 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:58:54,230 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. started... 2023-10-27 23:58:54,230 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f7ae7da0566a658f8f0ae5d6c67c438c 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:58:54,651 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228ab1b2734e32c4608a854acdfc366a5a9 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228ab1b2734e32c4608a854acdfc366a5a9 2023-10-27 23:58:55,059 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/.tmp/family1/ade366adf5ef43ec824cd6e595086303 2023-10-27 23:58:55,067 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/.tmp/family1/ade366adf5ef43ec824cd6e595086303 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/ade366adf5ef43ec824cd6e595086303 2023-10-27 23:58:55,073 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/ade366adf5ef43ec824cd6e595086303, entries=1, sequenceid=21, filesize=4.9 K 2023-10-27 23:58:55,074 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f7ae7da0566a658f8f0ae5d6c67c438c in 844ms, sequenceid=21, compaction requested=true 2023-10-27 23:58:55,074 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f7ae7da0566a658f8f0ae5d6c67c438c: 2023-10-27 23:58:55,074 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:58:55,074 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:55,074 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:55,075 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:55,075 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-10-27 23:58:55,075 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-10-27 23:58:55,111 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:55,111 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:55,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:55,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:55,112 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:55,112 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:55,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:55,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:55,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:55,117 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-10-27 23:58:55,117 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b280419[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:55,117 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-10-27 23:58:55,117 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,127 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,128 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-10-27 23:58:55,128 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-10-27 23:58:55,128 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:55,128 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:55,129 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:55,136 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:55,136 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-10-27 23:58:55,136 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:55,136 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-10-27 23:58:55,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:55,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:55,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:55,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:55,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:55,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:55,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:55,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:55,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:58:55,148 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:58:55,148 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-10-27 23:58:55,148 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:55,148 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:55,148 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,148 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:55,161 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,161 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,161 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:55,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:55,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:55,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:55,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:55,163 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:55,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:55,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:55,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,168 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:55,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:55,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:55,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:55,177 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:55,177 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:55,177 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,177 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:55,177 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-10-27 23:58:55,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:55,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:55,177 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:55,177 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,178 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:55,177 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-10-27 23:58:55,178 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:55,178 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:55,178 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:55,178 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:55,179 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:55,178 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,279 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:55,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:55,293 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:55,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-10-27 23:58:55,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,296 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-10-27 23:58:55,296 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:55,296 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-10-27 23:58:55,296 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-10-27 23:58:55,297 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,297 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,302 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:55,302 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:55,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:55,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:55,303 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:55,303 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:55,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,303 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:55,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,304 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,304 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-10-27 23:58:55,304 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:55,304 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-10-27 23:58:55,304 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:55,304 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-10-27 23:58:55,304 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:55,304 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:55,304 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:58:55,305 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. started... 2023-10-27 23:58:55,305 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f7ae7da0566a658f8f0ae5d6c67c438c 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:58:55,726 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231818a025023a74f9f906964e96a401d04 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231818a025023a74f9f906964e96a401d04 2023-10-27 23:58:56,134 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/.tmp/family1/b79bad75720845209ad4dac65a655024 2023-10-27 23:58:56,143 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/.tmp/family1/b79bad75720845209ad4dac65a655024 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/b79bad75720845209ad4dac65a655024 2023-10-27 23:58:56,150 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/b79bad75720845209ad4dac65a655024, entries=1, sequenceid=25, filesize=4.9 K 2023-10-27 23:58:56,151 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f7ae7da0566a658f8f0ae5d6c67c438c in 846ms, sequenceid=25, compaction requested=true 2023-10-27 23:58:56,151 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f7ae7da0566a658f8f0ae5d6c67c438c: 2023-10-27 23:58:56,151 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:58:56,151 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:56,151 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:56,151 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:56,151 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-10-27 23:58:56,152 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-10-27 23:58:56,161 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:56,161 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:56,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:56,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:56,161 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,162 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:56,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:56,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:56,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:56,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:56,165 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-10-27 23:58:56,165 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5e08791a[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:56,165 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-10-27 23:58:56,165 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,176 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,176 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-10-27 23:58:56,177 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-10-27 23:58:56,177 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:56,177 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:56,177 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:56,205 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-10-27 23:58:56,205 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:56,205 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:56,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:56,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:56,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:56,205 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-10-27 23:58:56,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:56,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:56,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:56,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:56,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:56,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:58:56,212 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:58:56,212 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-10-27 23:58:56,212 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:56,212 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:56,212 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,212 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:56,226 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,226 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:56,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:56,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,226 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:56,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:56,227 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:56,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:56,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:56,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:56,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,231 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:56,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:56,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:56,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:56,244 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:56,244 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:56,244 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,244 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-10-27 23:58:56,244 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:56,244 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-10-27 23:58:56,244 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,244 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:56,245 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:56,245 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:56,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:56,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:56,244 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:56,245 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:56,246 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,246 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,246 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:56,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:56,345 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:56,347 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:56,358 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:56,361 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-10-27 23:58:56,361 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,361 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-10-27 23:58:56,361 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:56,362 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-10-27 23:58:56,362 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-10-27 23:58:56,362 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,362 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,369 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:56,369 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:56,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:56,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:56,370 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:56,370 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:56,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,371 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:56,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,372 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,372 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-10-27 23:58:56,372 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:56,372 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-10-27 23:58:56,372 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:56,373 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-10-27 23:58:56,373 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:56,373 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:56,373 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:58:56,373 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. started... 2023-10-27 23:58:56,374 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f7ae7da0566a658f8f0ae5d6c67c438c 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:58:56,434 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-10-27 23:58:56,803 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010129216878b964404fb2551a6104427a64 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2016010129216878b964404fb2551a6104427a64 2023-10-27 23:58:57,213 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/.tmp/family1/ef1b5c7960cb4708ae152a10099bbc95 2023-10-27 23:58:57,226 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/.tmp/family1/ef1b5c7960cb4708ae152a10099bbc95 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/ef1b5c7960cb4708ae152a10099bbc95 2023-10-27 23:58:57,234 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/ef1b5c7960cb4708ae152a10099bbc95, entries=1, sequenceid=29, filesize=4.9 K 2023-10-27 23:58:57,235 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f7ae7da0566a658f8f0ae5d6c67c438c in 861ms, sequenceid=29, compaction requested=true 2023-10-27 23:58:57,235 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f7ae7da0566a658f8f0ae5d6c67c438c: 2023-10-27 23:58:57,235 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:58:57,235 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:57,236 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:57,236 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:57,236 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-10-27 23:58:57,236 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-10-27 23:58:57,269 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:57,269 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:57,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:57,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:57,270 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,270 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:57,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:57,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:57,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:57,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:57,272 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-10-27 23:58:57,272 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6c0d867a[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:57,272 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-10-27 23:58:57,273 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,277 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,277 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-10-27 23:58:57,277 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-10-27 23:58:57,277 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:57,277 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:57,278 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:57,285 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:57,285 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-10-27 23:58:57,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:57,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:57,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:57,286 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:57,286 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-10-27 23:58:57,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:57,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:57,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:57,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:57,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:57,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:58:57,290 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:58:57,290 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-10-27 23:58:57,291 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:57,291 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:57,291 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,291 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:57,302 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,302 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,302 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:57,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:57,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:57,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:57,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:57,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:57,304 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:57,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:57,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,307 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:57,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:57,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:57,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:57,319 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:57,319 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:57,319 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:57,319 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:57,319 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-10-27 23:58:57,319 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,320 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,319 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-10-27 23:58:57,320 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:57,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:57,320 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:57,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:57,320 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,320 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,320 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:57,320 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,320 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:57,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:57,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:57,421 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:57,422 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:57,430 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:57,433 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-10-27 23:58:57,433 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,433 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-10-27 23:58:57,433 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:57,433 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-10-27 23:58:57,433 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-10-27 23:58:57,434 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,434 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,444 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:57,444 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:57,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:57,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:57,444 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:57,444 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:57,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,445 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:57,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,445 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,446 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-10-27 23:58:57,446 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:57,446 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-10-27 23:58:57,446 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:57,446 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-10-27 23:58:57,446 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:57,446 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:57,446 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:58:57,446 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. started... 2023-10-27 23:58:57,446 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f7ae7da0566a658f8f0ae5d6c67c438c 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:58:57,870 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010316cfb4584fc64cdc860d387084c3f739 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2016010316cfb4584fc64cdc860d387084c3f739 2023-10-27 23:58:58,278 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/.tmp/family1/542596e03bc04811a1c499fe5ae13625 2023-10-27 23:58:58,293 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/.tmp/family1/542596e03bc04811a1c499fe5ae13625 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/542596e03bc04811a1c499fe5ae13625 2023-10-27 23:58:58,299 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/542596e03bc04811a1c499fe5ae13625, entries=1, sequenceid=33, filesize=4.9 K 2023-10-27 23:58:58,300 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f7ae7da0566a658f8f0ae5d6c67c438c in 854ms, sequenceid=33, compaction requested=true 2023-10-27 23:58:58,300 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f7ae7da0566a658f8f0ae5d6c67c438c: 2023-10-27 23:58:58,300 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:58:58,300 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:58,300 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:58,300 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:58,301 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-10-27 23:58:58,301 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-10-27 23:58:58,336 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:58,336 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:58,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:58,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:58,337 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,337 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:58,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:58,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:58,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:58,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:58,342 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-10-27 23:58:58,342 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@26817f6a[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:58,342 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-10-27 23:58:58,342 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,352 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,353 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-10-27 23:58:58,353 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-10-27 23:58:58,353 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:58,353 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:58,354 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:58,361 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:58,361 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-10-27 23:58:58,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:58,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:58,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:58,361 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:58,362 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-10-27 23:58:58,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:58,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:58,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:58,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:58,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:58,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:58:58,371 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:58:58,372 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-10-27 23:58:58,372 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:58,372 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:58,372 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,372 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:58,384 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,384 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,384 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:58,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:58,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:58,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:58,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:58,386 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:58,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:58,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:58,389 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:58,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:58,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:58,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:58,402 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:58,402 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:58,402 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:58,402 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-10-27 23:58:58,403 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:58,402 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:58,402 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:58,403 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-10-27 23:58:58,403 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:58,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:58,402 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,403 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:58,404 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:58,404 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,404 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,404 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:58,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:58,504 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:58,505 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:58,511 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:58,514 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-10-27 23:58:58,514 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,514 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-10-27 23:58:58,514 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:58,514 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-10-27 23:58:58,514 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-10-27 23:58:58,515 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,515 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,519 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:58,519 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:58,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:58,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:58,520 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:58,520 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:58,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,521 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:58,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,522 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,522 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-10-27 23:58:58,522 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:58,522 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-10-27 23:58:58,522 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:58,522 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-10-27 23:58:58,522 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:58,523 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:58,523 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:58:58,523 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. started... 2023-10-27 23:58:58,523 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f7ae7da0566a658f8f0ae5d6c67c438c 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:58:58,947 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090749a60db371034d44986784c5d57d8e7c to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015090749a60db371034d44986784c5d57d8e7c 2023-10-27 23:58:59,357 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/.tmp/family1/7ddaaf33ba6343908fc9d3fc407014e7 2023-10-27 23:58:59,372 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/.tmp/family1/7ddaaf33ba6343908fc9d3fc407014e7 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/7ddaaf33ba6343908fc9d3fc407014e7 2023-10-27 23:58:59,380 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/7ddaaf33ba6343908fc9d3fc407014e7, entries=1, sequenceid=37, filesize=4.9 K 2023-10-27 23:58:59,381 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f7ae7da0566a658f8f0ae5d6c67c438c in 858ms, sequenceid=37, compaction requested=true 2023-10-27 23:58:59,381 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f7ae7da0566a658f8f0ae5d6c67c438c: 2023-10-27 23:58:59,381 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:58:59,381 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:58:59,381 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:58:59,381 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:59,381 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-10-27 23:58:59,381 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-10-27 23:58:59,410 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:59,410 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:59,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:59,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:59,411 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,411 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:58:59,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:59,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:59,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:59,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:59,412 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-10-27 23:58:59,412 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7d52bf6f[Count = 0] remaining members to acquire global barrier 2023-10-27 23:58:59,412 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-10-27 23:58:59,412 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,438 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,439 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-10-27 23:58:59,439 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-10-27 23:58:59,439 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:58:59,439 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:59,439 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:58:59,452 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:59,452 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-10-27 23:58:59,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:59,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:59,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:59,452 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:59,452 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-10-27 23:58:59,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:59,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:59,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:59,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:59,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:59,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:58:59,456 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:58:59,456 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-10-27 23:58:59,457 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:58:59,457 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:58:59,457 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,457 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:58:59,467 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,467 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,467 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:59,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:58:59,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:58:59,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:59,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:59,469 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:59,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:58:59,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:58:59,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,471 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:59,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:59,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:58:59,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:59,477 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:59,477 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,477 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,477 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:59,477 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:59,477 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,477 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-10-27 23:58:59,478 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:58:59,477 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:58:59,478 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,478 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,478 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:58:59,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:59,478 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:58:59,479 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:58:59,478 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-10-27 23:58:59,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:59,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:58:59,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:58:59,579 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:58:59,581 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:58:59,592 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:58:59,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-10-27 23:58:59,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,596 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-10-27 23:58:59,596 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:59,596 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-10-27 23:58:59,596 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-10-27 23:58:59,598 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,598 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,602 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:58:59,602 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:59,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:58:59,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:58:59,603 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:58:59,603 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:58:59,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,604 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:58:59,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,606 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,606 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-10-27 23:58:59,606 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-10-27 23:58:59,606 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-10-27 23:58:59,606 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:58:59,607 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-10-27 23:58:59,607 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:58:59,607 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:58:59,607 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:58:59,607 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. started... 2023-10-27 23:58:59,607 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f7ae7da0566a658f8f0ae5d6c67c438c 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:59:00,035 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120f230851b8dc840cd8af18d1f8a9fce71 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151120f230851b8dc840cd8af18d1f8a9fce71 2023-10-27 23:59:00,443 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/.tmp/family1/742ca8abd8514815868f34164ef4a165 2023-10-27 23:59:00,455 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/.tmp/family1/742ca8abd8514815868f34164ef4a165 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/742ca8abd8514815868f34164ef4a165 2023-10-27 23:59:00,462 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/742ca8abd8514815868f34164ef4a165, entries=1, sequenceid=41, filesize=4.9 K 2023-10-27 23:59:00,464 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f7ae7da0566a658f8f0ae5d6c67c438c in 857ms, sequenceid=41, compaction requested=true 2023-10-27 23:59:00,464 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f7ae7da0566a658f8f0ae5d6c67c438c: 2023-10-27 23:59:00,464 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:59:00,464 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:00,464 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:00,464 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:00,464 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-10-27 23:59:00,464 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-10-27 23:59:00,494 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:00,494 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:59:00,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:00,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:00,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:00,494 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:59:00,494 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:00,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:00,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:00,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:59:00,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:00,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:00,497 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-10-27 23:59:00,497 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@608fc976[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:00,497 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-10-27 23:59:00,497 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:59:00,502 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:59:00,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:59:00,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:59:00,502 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-10-27 23:59:00,502 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-10-27 23:59:00,502 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:00,503 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:00,503 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:00,510 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-10-27 23:59:00,510 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:00,511 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:00,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:00,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:00,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:00,511 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-10-27 23:59:00,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:00,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:00,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:59:00,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:00,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:00,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:59:00,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:00,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:00,517 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:59:00,517 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-10-27 23:59:00,517 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:00,517 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:00,518 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-10-27 23:59:00,518 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:00,527 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:59:00,527 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:59:00,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:59:00,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:59:00,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:59:00,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:00,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:00,527 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:00,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:00,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:00,529 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:00,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:00,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:59:00,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:59:00,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:00,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:59:00,532 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:00,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:00,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:00,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-10-27 23:59:00,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:00,544 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:00,544 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:59:00,544 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:59:00,544 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:00,544 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:00,544 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-10-27 23:59:00,544 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:00,544 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:00,544 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:59:00,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:00,545 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-10-27 23:59:00,544 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:00,545 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:00,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:00,545 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:59:00,546 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:00,546 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-10-27 23:59:00,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:00,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:00,646 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:00,648 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:00,651 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36413] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-10-27 23:59:00,654 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] procedure2.ProcedureExecutor(1028): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-10-27 23:59:00,656 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-10-27 23:59:00,657 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-27 23:59:00,657 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698451740657 2023-10-27 23:59:00,660 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-10-27 23:59:00,662 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-10-27 23:59:00,662 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-10-27 23:59:00,664 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] zookeeper.ReadOnlyZKClient(139): Connect 0x065c1da2 to 127.0.0.1:57369 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-10-27 23:59:00,678 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@17182b39, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-27 23:59:00,678 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-10-27 23:59:01,530 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103bc2106b3a5ec44b5b2624c940e66ecfe to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103bc2106b3a5ec44b5b2624c940e66ecfe 2023-10-27 23:59:01,531 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-10-27 23:59:01,544 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-10-27 23:59:01,545 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:39162, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-10-27 23:59:01,553 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-10-27 23:59:01,555 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:50554, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-10-27 23:59:01,566 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dd4c679fadb9485c95b878d03c2cf9c2 first=Optional[r3] last=Optional[r6] 2023-10-27 23:59:01,567 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dd4c679fadb9485c95b878d03c2cf9c2}] 2023-10-27 23:59:01,573 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dd4c679fadb9485c95b878d03c2cf9c2 for inclusion in f7ae7da0566a658f8f0ae5d6c67c438c/family1 2023-10-27 23:59:01,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-10-27 23:59:01,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(777): Region bounds: first= last= 2023-10-27 23:59:01,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for f7ae7da0566a658f8f0ae5d6c67c438c: 2023-10-27 23:59:01,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dd4c679fadb9485c95b878d03c2cf9c2 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMinorCompactionWithWeeklyPolicy__cab0ee28913mkhn9r6rf1klc49g69b3qbm1c6pom4ourujiatjhloti7n64d6jbm/family1/dd4c679fadb9485c95b878d03c2cf9c2 2023-10-27 23:59:01,583 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMinorCompactionWithWeeklyPolicy__cab0ee28913mkhn9r6rf1klc49g69b3qbm1c6pom4ourujiatjhloti7n64d6jbm/family1/dd4c679fadb9485c95b878d03c2cf9c2 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/f53d3b89a7ff492c8538784b99dd53e2_SeqId_44_ 2023-10-27 23:59:01,584 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMinorCompactionWithWeeklyPolicy__cab0ee28913mkhn9r6rf1klc49g69b3qbm1c6pom4ourujiatjhloti7n64d6jbm/family1/dd4c679fadb9485c95b878d03c2cf9c2 into f7ae7da0566a658f8f0ae5d6c67c438c/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/f53d3b89a7ff492c8538784b99dd53e2_SeqId_44_ - updating store file list. 2023-10-27 23:59:01,591 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/f53d3b89a7ff492c8538784b99dd53e2_SeqId_44_ into f7ae7da0566a658f8f0ae5d6c67c438c/family1 2023-10-27 23:59:01,591 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMinorCompactionWithWeeklyPolicy__cab0ee28913mkhn9r6rf1klc49g69b3qbm1c6pom4ourujiatjhloti7n64d6jbm/family1/dd4c679fadb9485c95b878d03c2cf9c2 into f7ae7da0566a658f8f0ae5d6c67c438c/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/f53d3b89a7ff492c8538784b99dd53e2_SeqId_44_) 2023-10-27 23:59:01,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMinorCompactionWithWeeklyPolicy__cab0ee28913mkhn9r6rf1klc49g69b3qbm1c6pom4ourujiatjhloti7n64d6jbm/family1/dd4c679fadb9485c95b878d03c2cf9c2 2023-10-27 23:59:01,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-10-27 23:59:01,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-10-27 23:59:01,597 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(1912): f7ae7da0566a658f8f0ae5d6c67c438c/family1 is initiating minor compaction 2023-10-27 23:59:01,597 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@5ba8acb1; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-27 23:59:01,597 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of f7ae7da0566a658f8f0ae5d6c67c438c/family1 in testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:59:01,597 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/c84ccc42ea274ead85b173f0d69c71d3, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/3920e0e4eeae4d4396c3b00722ac1475, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/0655698fba01487f87e9cfa70c206384, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/ade366adf5ef43ec824cd6e595086303, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/b79bad75720845209ad4dac65a655024, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/ef1b5c7960cb4708ae152a10099bbc95, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/542596e03bc04811a1c499fe5ae13625, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/7ddaaf33ba6343908fc9d3fc407014e7, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/742ca8abd8514815868f34164ef4a165, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/f53d3b89a7ff492c8538784b99dd53e2_SeqId_44_] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/.tmp, totalSize=50.0 K 2023-10-27 23:59:01,597 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting c84ccc42ea274ead85b173f0d69c71d3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-10-27 23:59:01,597 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 3920e0e4eeae4d4396c3b00722ac1475, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-10-27 23:59:01,597 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 0655698fba01487f87e9cfa70c206384, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-10-27 23:59:01,597 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting ade366adf5ef43ec824cd6e595086303, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-10-27 23:59:01,597 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting b79bad75720845209ad4dac65a655024, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-10-27 23:59:01,598 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting ef1b5c7960cb4708ae152a10099bbc95, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-10-27 23:59:01,598 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 542596e03bc04811a1c499fe5ae13625, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-10-27 23:59:01,598 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 7ddaaf33ba6343908fc9d3fc407014e7, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-10-27 23:59:01,598 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 742ca8abd8514815868f34164ef4a165, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-10-27 23:59:01,598 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting f53d3b89a7ff492c8538784b99dd53e2_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-10-27 23:59:01,599 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-10-27 23:59:01,600 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:59:01,601 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228ab1b2734e32c4608a854acdfc366a5a9 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228ab1b2734e32c4608a854acdfc366a5a9 2023-10-27 23:59:01,603 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231818a025023a74f9f906964e96a401d04 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231818a025023a74f9f906964e96a401d04 2023-10-27 23:59:01,604 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2016010129216878b964404fb2551a6104427a64 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2016010129216878b964404fb2551a6104427a64 2023-10-27 23:59:01,605 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2016010316cfb4584fc64cdc860d387084c3f739 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2016010316cfb4584fc64cdc860d387084c3f739 2023-10-27 23:59:01,605 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-10-27 23:59:01,605 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-10-27 23:59:01,606 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-10-27 23:59:01,606 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-10-27 23:59:01,606 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-10-27 23:59:01,606 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-10-27 23:59:01,606 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-10-27 23:59:01,606 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-10-27 23:59:01,606 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-10-27 23:59:01,607 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-10-27 23:59:01,624 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] throttle.PressureAwareThroughputController(145): f7ae7da0566a658f8f0ae5d6c67c438c#family1#compaction#34 average throughput is 1.36 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-10-27 23:59:02,450 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205dbcb845fc3804087aceb5431b2d3f880 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205dbcb845fc3804087aceb5431b2d3f880 2023-10-27 23:59:02,451 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/.tmp/family1/04dcd7d5fc9746a68185f2b00566ca5d as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/04dcd7d5fc9746a68185f2b00566ca5d 2023-10-27 23:59:02,451 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-10-27 23:59:02,458 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in f7ae7da0566a658f8f0ae5d6c67c438c/family1 of f7ae7da0566a658f8f0ae5d6c67c438c into 04dcd7d5fc9746a68185f2b00566ca5d(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-10-27 23:59:02,459 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for f7ae7da0566a658f8f0ae5d6c67c438c: 2023-10-27 23:59:02,459 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1698451141596; duration=0sec 2023-10-27 23:59:02,459 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-27 23:59:02,481 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/4e2df68e243748ce9afe7056fb5a30ba first=Optional[r1] last=Optional[r2] 2023-10-27 23:59:02,481 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/4e2df68e243748ce9afe7056fb5a30ba}] 2023-10-27 23:59:02,484 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/4e2df68e243748ce9afe7056fb5a30ba for inclusion in f7ae7da0566a658f8f0ae5d6c67c438c/family1 2023-10-27 23:59:02,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-10-27 23:59:02,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(777): Region bounds: first= last= 2023-10-27 23:59:02,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for f7ae7da0566a658f8f0ae5d6c67c438c: 2023-10-27 23:59:02,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/4e2df68e243748ce9afe7056fb5a30ba to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMinorCompactionWithWeeklyPolicy__hjgtvntfbe4aru1cslj1ok2pifverqk7bslv68d3ao83rjfs6rg8ogc9aao9te45/family1/4e2df68e243748ce9afe7056fb5a30ba 2023-10-27 23:59:02,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMinorCompactionWithWeeklyPolicy__hjgtvntfbe4aru1cslj1ok2pifverqk7bslv68d3ao83rjfs6rg8ogc9aao9te45/family1/4e2df68e243748ce9afe7056fb5a30ba as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/a884f28878984cbea3919710ead51594_SeqId_47_ 2023-10-27 23:59:02,494 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMinorCompactionWithWeeklyPolicy__hjgtvntfbe4aru1cslj1ok2pifverqk7bslv68d3ao83rjfs6rg8ogc9aao9te45/family1/4e2df68e243748ce9afe7056fb5a30ba into f7ae7da0566a658f8f0ae5d6c67c438c/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/a884f28878984cbea3919710ead51594_SeqId_47_ - updating store file list. 2023-10-27 23:59:02,498 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/a884f28878984cbea3919710ead51594_SeqId_47_ into f7ae7da0566a658f8f0ae5d6c67c438c/family1 2023-10-27 23:59:02,498 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMinorCompactionWithWeeklyPolicy__hjgtvntfbe4aru1cslj1ok2pifverqk7bslv68d3ao83rjfs6rg8ogc9aao9te45/family1/4e2df68e243748ce9afe7056fb5a30ba into f7ae7da0566a658f8f0ae5d6c67c438c/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/a884f28878984cbea3919710ead51594_SeqId_47_) 2023-10-27 23:59:02,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMinorCompactionWithWeeklyPolicy__hjgtvntfbe4aru1cslj1ok2pifverqk7bslv68d3ao83rjfs6rg8ogc9aao9te45/family1/4e2df68e243748ce9afe7056fb5a30ba 2023-10-27 23:59:02,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-10-27 23:59:02,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 181014 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-10-27 23:59:02,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(1912): f7ae7da0566a658f8f0ae5d6c67c438c/family1 is initiating minor compaction (all files) 2023-10-27 23:59:02,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@f211de7; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-27 23:59:02,500 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of f7ae7da0566a658f8f0ae5d6c67c438c/family1 in testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-27 23:59:02,500 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/1b66eb8acac8414a96612d5ace3103df, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/04dcd7d5fc9746a68185f2b00566ca5d, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/a884f28878984cbea3919710ead51594_SeqId_47_] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/.tmp, totalSize=176.8 K 2023-10-27 23:59:02,501 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 1b66eb8acac8414a96612d5ace3103df, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878821089 2023-10-27 23:59:02,501 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 04dcd7d5fc9746a68185f2b00566ca5d, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621221089 2023-10-27 23:59:02,502 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting a884f28878984cbea3919710ead51594_SeqId_47_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965221089 2023-10-27 23:59:02,502 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-10-27 23:59:02,503 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:59:02,504 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201e8c7717728d444aeb7f5741f11c1d65c to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201e8c7717728d444aeb7f5741f11c1d65c 2023-10-27 23:59:02,506 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205411ad5847b5a430dbe5a2d15f7ea4e55 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205411ad5847b5a430dbe5a2d15f7ea4e55 2023-10-27 23:59:02,506 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-10-27 23:59:02,506 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-10-27 23:59:02,506 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-10-27 23:59:02,507 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-10-27 23:59:02,507 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-27 23:59:02,508 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-10-27 23:59:02,509 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.8560 sec 2023-10-27 23:59:02,519 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] throttle.PressureAwareThroughputController(145): f7ae7da0566a658f8f0ae5d6c67c438c#family1#compaction#35 average throughput is 22.09 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-10-27 23:59:02,538 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/c84ccc42ea274ead85b173f0d69c71d3, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/3920e0e4eeae4d4396c3b00722ac1475, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/0655698fba01487f87e9cfa70c206384, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/ade366adf5ef43ec824cd6e595086303, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/b79bad75720845209ad4dac65a655024, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/ef1b5c7960cb4708ae152a10099bbc95, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/542596e03bc04811a1c499fe5ae13625, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/7ddaaf33ba6343908fc9d3fc407014e7, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/742ca8abd8514815868f34164ef4a165, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/f53d3b89a7ff492c8538784b99dd53e2_SeqId_44_] to archive 2023-10-27 23:59:02,539 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:59:02,540 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/c84ccc42ea274ead85b173f0d69c71d3 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/c84ccc42ea274ead85b173f0d69c71d3 2023-10-27 23:59:02,543 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/3920e0e4eeae4d4396c3b00722ac1475 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/3920e0e4eeae4d4396c3b00722ac1475 2023-10-27 23:59:02,545 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/0655698fba01487f87e9cfa70c206384 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/0655698fba01487f87e9cfa70c206384 2023-10-27 23:59:02,546 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/ade366adf5ef43ec824cd6e595086303 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/ade366adf5ef43ec824cd6e595086303 2023-10-27 23:59:02,547 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/b79bad75720845209ad4dac65a655024 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/b79bad75720845209ad4dac65a655024 2023-10-27 23:59:02,549 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/ef1b5c7960cb4708ae152a10099bbc95 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/ef1b5c7960cb4708ae152a10099bbc95 2023-10-27 23:59:02,550 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/542596e03bc04811a1c499fe5ae13625 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/542596e03bc04811a1c499fe5ae13625 2023-10-27 23:59:02,552 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/7ddaaf33ba6343908fc9d3fc407014e7 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/7ddaaf33ba6343908fc9d3fc407014e7 2023-10-27 23:59:02,553 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/742ca8abd8514815868f34164ef4a165 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/742ca8abd8514815868f34164ef4a165 2023-10-27 23:59:02,554 DEBUG [hfile_cleaner-dir-scan-pool-1] cleaner.CleanerChore(483): Couldn't delete 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithWeeklyPolicy' yet because it isn't empty w/exception. org.apache.hadoop.fs.PathIsNotEmptyDirectoryException: ``/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithWeeklyPolicy is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) ': Directory is not empty: `/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithWeeklyPolicy is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1595) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:881) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:878) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:878) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$4(CleanerChore.java:453) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.deleteAction(CleanerChore.java:478) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:453) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:792) at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2153) at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:61) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:439) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$2(CleanerChore.java:432) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): `/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithWeeklyPolicy is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1540) at org.apache.hadoop.ipc.Client.call(Client.java:1486) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy29.delete(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:572) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy30.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1593) ... 29 more 2023-10-27 23:59:02,555 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/f53d3b89a7ff492c8538784b99dd53e2_SeqId_44_ to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/f53d3b89a7ff492c8538784b99dd53e2_SeqId_44_ 2023-10-27 23:59:02,623 INFO [Listener at localhost/40295] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=263 (was 265), OpenFileDescriptor=627 (was 632), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=229 (was 250), ProcessCount=167 (was 167), AvailableMemoryMB=4889 (was 4972) 2023-10-27 23:59:02,633 INFO [Listener at localhost/40295] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=263, OpenFileDescriptor=627, MaxFileDescriptor=60000, SystemLoadAverage=229, ProcessCount=167, AvailableMemoryMB=4888 2023-10-27 23:59:02,635 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.HMaster$4(2112): Client=jenkins//188.40.62.62 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-10-27 23:59:02,636 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure2.ProcedureExecutor(1028): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:02,638 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-10-27 23:59:02,638 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(700): Client=jenkins//188.40.62.62 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-10-27 23:59:02,639 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-10-27 23:59:02,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=27 2023-10-27 23:59:02,641 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testScannerOnBulkLoadRefHFiles/db3340090786ac1f09ea9c45bcb224a0 2023-10-27 23:59:02,641 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testScannerOnBulkLoadRefHFiles/a91badaa245ed444aa843d100a54dfca 2023-10-27 23:59:02,641 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403 2023-10-27 23:59:02,641 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testScannerOnBulkLoadRefHFiles/db3340090786ac1f09ea9c45bcb224a0 empty. 2023-10-27 23:59:02,641 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testScannerOnBulkLoadRefHFiles/a91badaa245ed444aa843d100a54dfca empty. 2023-10-27 23:59:02,641 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403 empty. 2023-10-27 23:59:02,642 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testScannerOnBulkLoadRefHFiles/db3340090786ac1f09ea9c45bcb224a0 2023-10-27 23:59:02,642 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testScannerOnBulkLoadRefHFiles/a91badaa245ed444aa843d100a54dfca 2023-10-27 23:59:02,642 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403 2023-10-27 23:59:02,642 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-10-27 23:59:02,656 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-10-27 23:59:02,657 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7675): creating {ENCODED => db3340090786ac1f09ea9c45bcb224a0, NAME => 'testScannerOnBulkLoadRefHFiles,,1698451142635.db3340090786ac1f09ea9c45bcb224a0.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp 2023-10-27 23:59:02,658 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7675): creating {ENCODED => a91badaa245ed444aa843d100a54dfca, NAME => 'testScannerOnBulkLoadRefHFiles,1,1698451142635.a91badaa245ed444aa843d100a54dfca.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp 2023-10-27 23:59:02,658 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7675): creating {ENCODED => f0c80b795ea0c9c5ec8ad26a6060e403, NAME => 'testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp 2023-10-27 23:59:02,670 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1698451142635.a91badaa245ed444aa843d100a54dfca.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:59:02,670 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1604): Closing a91badaa245ed444aa843d100a54dfca, disabling compactions & flushes 2023-10-27 23:59:02,670 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1698451142635.a91badaa245ed444aa843d100a54dfca. 2023-10-27 23:59:02,670 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1698451142635.a91badaa245ed444aa843d100a54dfca. 2023-10-27 23:59:02,670 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1698451142635.a91badaa245ed444aa843d100a54dfca. after waiting 0 ms 2023-10-27 23:59:02,670 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1698451142635.a91badaa245ed444aa843d100a54dfca. 2023-10-27 23:59:02,670 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1698451142635.a91badaa245ed444aa843d100a54dfca. 2023-10-27 23:59:02,670 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1558): Region close journal for a91badaa245ed444aa843d100a54dfca: 2023-10-27 23:59:02,740 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=27 2023-10-27 23:59:02,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=27 2023-10-27 23:59:03,068 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1698451142635.db3340090786ac1f09ea9c45bcb224a0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:59:03,069 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1604): Closing db3340090786ac1f09ea9c45bcb224a0, disabling compactions & flushes 2023-10-27 23:59:03,069 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1698451142635.db3340090786ac1f09ea9c45bcb224a0. 2023-10-27 23:59:03,069 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1698451142635.db3340090786ac1f09ea9c45bcb224a0. 2023-10-27 23:59:03,069 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1698451142635.db3340090786ac1f09ea9c45bcb224a0. after waiting 0 ms 2023-10-27 23:59:03,069 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1698451142635.db3340090786ac1f09ea9c45bcb224a0. 2023-10-27 23:59:03,069 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:59:03,069 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1698451142635.db3340090786ac1f09ea9c45bcb224a0. 2023-10-27 23:59:03,070 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1558): Region close journal for db3340090786ac1f09ea9c45bcb224a0: 2023-10-27 23:59:03,070 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1604): Closing f0c80b795ea0c9c5ec8ad26a6060e403, disabling compactions & flushes 2023-10-27 23:59:03,070 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403. 2023-10-27 23:59:03,070 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403. 2023-10-27 23:59:03,070 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403. after waiting 0 ms 2023-10-27 23:59:03,070 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403. 2023-10-27 23:59:03,071 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403. 2023-10-27 23:59:03,071 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1558): Region close journal for f0c80b795ea0c9c5ec8ad26a6060e403: 2023-10-27 23:59:03,076 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-10-27 23:59:03,078 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1698451142635.a91badaa245ed444aa843d100a54dfca.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1698451143077"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451143077"}]},"ts":"1698451143077"} 2023-10-27 23:59:03,078 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1698451142635.db3340090786ac1f09ea9c45bcb224a0.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1698451143077"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451143077"}]},"ts":"1698451143077"} 2023-10-27 23:59:03,078 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1698451143077"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451143077"}]},"ts":"1698451143077"} 2023-10-27 23:59:03,082 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-10-27 23:59:03,083 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-10-27 23:59:03,083 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451143083"}]},"ts":"1698451143083"} 2023-10-27 23:59:03,084 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-10-27 23:59:03,136 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=db3340090786ac1f09ea9c45bcb224a0, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=a91badaa245ed444aa843d100a54dfca, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=f0c80b795ea0c9c5ec8ad26a6060e403, ASSIGN}] 2023-10-27 23:59:03,140 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=f0c80b795ea0c9c5ec8ad26a6060e403, ASSIGN 2023-10-27 23:59:03,140 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=a91badaa245ed444aa843d100a54dfca, ASSIGN 2023-10-27 23:59:03,140 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=db3340090786ac1f09ea9c45bcb224a0, ASSIGN 2023-10-27 23:59:03,141 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=f0c80b795ea0c9c5ec8ad26a6060e403, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,43469,1698451096272; forceNewPlan=false, retain=false 2023-10-27 23:59:03,142 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=a91badaa245ed444aa843d100a54dfca, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,43469,1698451096272; forceNewPlan=false, retain=false 2023-10-27 23:59:03,142 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=db3340090786ac1f09ea9c45bcb224a0, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,43469,1698451096272; forceNewPlan=false, retain=false 2023-10-27 23:59:03,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=27 2023-10-27 23:59:03,295 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=f0c80b795ea0c9c5ec8ad26a6060e403, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:03,295 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=db3340090786ac1f09ea9c45bcb224a0, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:03,295 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=a91badaa245ed444aa843d100a54dfca, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:03,296 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1698451142635.db3340090786ac1f09ea9c45bcb224a0.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1698451143295"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451143295"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451143295"}]},"ts":"1698451143295"} 2023-10-27 23:59:03,296 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1698451143295"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451143295"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451143295"}]},"ts":"1698451143295"} 2023-10-27 23:59:03,296 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1698451142635.a91badaa245ed444aa843d100a54dfca.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1698451143295"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451143295"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451143295"}]},"ts":"1698451143295"} 2023-10-27 23:59:03,299 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE; OpenRegionProcedure db3340090786ac1f09ea9c45bcb224a0, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-27 23:59:03,301 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=32, ppid=30, state=RUNNABLE; OpenRegionProcedure f0c80b795ea0c9c5ec8ad26a6060e403, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-27 23:59:03,302 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=33, ppid=29, state=RUNNABLE; OpenRegionProcedure a91badaa245ed444aa843d100a54dfca, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-27 23:59:03,353 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/.tmp/family1/429e4b5c29f7472bb2042929e8a57b92 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/429e4b5c29f7472bb2042929e8a57b92 2023-10-27 23:59:03,362 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in f7ae7da0566a658f8f0ae5d6c67c438c/family1 of f7ae7da0566a658f8f0ae5d6c67c438c into 429e4b5c29f7472bb2042929e8a57b92(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-10-27 23:59:03,363 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for f7ae7da0566a658f8f0ae5d6c67c438c: 2023-10-27 23:59:03,363 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c., storeName=family1, fileCount=3, fileSize=176.8 K (165.1 K, 6.6 K, 5.2 K), priority=2, time=1698451142500; duration=0sec 2023-10-27 23:59:03,363 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-27 23:59:03,435 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/1b66eb8acac8414a96612d5ace3103df, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/04dcd7d5fc9746a68185f2b00566ca5d, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/a884f28878984cbea3919710ead51594_SeqId_47_] to archive 2023-10-27 23:59:03,436 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:59:03,441 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/1b66eb8acac8414a96612d5ace3103df to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/1b66eb8acac8414a96612d5ace3103df 2023-10-27 23:59:03,444 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/04dcd7d5fc9746a68185f2b00566ca5d to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/04dcd7d5fc9746a68185f2b00566ca5d 2023-10-27 23:59:03,447 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/a884f28878984cbea3919710ead51594_SeqId_47_ to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/family1/a884f28878984cbea3919710ead51594_SeqId_47_ 2023-10-27 23:59:03,453 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:03,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open testScannerOnBulkLoadRefHFiles,,1698451142635.db3340090786ac1f09ea9c45bcb224a0. 2023-10-27 23:59:03,458 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => db3340090786ac1f09ea9c45bcb224a0, NAME => 'testScannerOnBulkLoadRefHFiles,,1698451142635.db3340090786ac1f09ea9c45bcb224a0.', STARTKEY => '', ENDKEY => '1'} 2023-10-27 23:59:03,459 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles db3340090786ac1f09ea9c45bcb224a0 2023-10-27 23:59:03,459 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1698451142635.db3340090786ac1f09ea9c45bcb224a0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:59:03,459 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for db3340090786ac1f09ea9c45bcb224a0 2023-10-27 23:59:03,459 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for db3340090786ac1f09ea9c45bcb224a0 2023-10-27 23:59:03,462 INFO [StoreOpener-db3340090786ac1f09ea9c45bcb224a0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region db3340090786ac1f09ea9c45bcb224a0 2023-10-27 23:59:03,465 DEBUG [StoreOpener-db3340090786ac1f09ea9c45bcb224a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/db3340090786ac1f09ea9c45bcb224a0/family1 2023-10-27 23:59:03,465 DEBUG [StoreOpener-db3340090786ac1f09ea9c45bcb224a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/db3340090786ac1f09ea9c45bcb224a0/family1 2023-10-27 23:59:03,466 INFO [StoreOpener-db3340090786ac1f09ea9c45bcb224a0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region db3340090786ac1f09ea9c45bcb224a0 columnFamilyName family1 2023-10-27 23:59:03,469 INFO [StoreOpener-db3340090786ac1f09ea9c45bcb224a0-1] regionserver.HStore(310): Store=db3340090786ac1f09ea9c45bcb224a0/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-27 23:59:03,470 INFO [StoreOpener-db3340090786ac1f09ea9c45bcb224a0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region db3340090786ac1f09ea9c45bcb224a0 2023-10-27 23:59:03,472 DEBUG [StoreOpener-db3340090786ac1f09ea9c45bcb224a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/db3340090786ac1f09ea9c45bcb224a0/family2 2023-10-27 23:59:03,472 DEBUG [StoreOpener-db3340090786ac1f09ea9c45bcb224a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/db3340090786ac1f09ea9c45bcb224a0/family2 2023-10-27 23:59:03,472 INFO [StoreOpener-db3340090786ac1f09ea9c45bcb224a0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region db3340090786ac1f09ea9c45bcb224a0 columnFamilyName family2 2023-10-27 23:59:03,474 INFO [StoreOpener-db3340090786ac1f09ea9c45bcb224a0-1] regionserver.HStore(310): Store=db3340090786ac1f09ea9c45bcb224a0/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-27 23:59:03,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/db3340090786ac1f09ea9c45bcb224a0 2023-10-27 23:59:03,476 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/db3340090786ac1f09ea9c45bcb224a0 2023-10-27 23:59:03,478 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-10-27 23:59:03,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for db3340090786ac1f09ea9c45bcb224a0 2023-10-27 23:59:03,482 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/db3340090786ac1f09ea9c45bcb224a0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-10-27 23:59:03,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened db3340090786ac1f09ea9c45bcb224a0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69320586, jitterRate=0.032957226037979126}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-10-27 23:59:03,483 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for db3340090786ac1f09ea9c45bcb224a0: 2023-10-27 23:59:03,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1698451142635.db3340090786ac1f09ea9c45bcb224a0., pid=31, masterSystemTime=1698451143453 2023-10-27 23:59:03,486 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1698451142635.db3340090786ac1f09ea9c45bcb224a0. 2023-10-27 23:59:03,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened testScannerOnBulkLoadRefHFiles,,1698451142635.db3340090786ac1f09ea9c45bcb224a0. 2023-10-27 23:59:03,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403. 2023-10-27 23:59:03,487 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f0c80b795ea0c9c5ec8ad26a6060e403, NAME => 'testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403.', STARTKEY => '2', ENDKEY => ''} 2023-10-27 23:59:03,487 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles f0c80b795ea0c9c5ec8ad26a6060e403 2023-10-27 23:59:03,487 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=db3340090786ac1f09ea9c45bcb224a0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:03,487 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:59:03,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for f0c80b795ea0c9c5ec8ad26a6060e403 2023-10-27 23:59:03,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for f0c80b795ea0c9c5ec8ad26a6060e403 2023-10-27 23:59:03,488 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1698451142635.db3340090786ac1f09ea9c45bcb224a0.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1698451143487"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451143487"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451143487"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451143487"}]},"ts":"1698451143487"} 2023-10-27 23:59:03,490 INFO [StoreOpener-f0c80b795ea0c9c5ec8ad26a6060e403-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f0c80b795ea0c9c5ec8ad26a6060e403 2023-10-27 23:59:03,492 DEBUG [StoreOpener-f0c80b795ea0c9c5ec8ad26a6060e403-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/family1 2023-10-27 23:59:03,492 DEBUG [StoreOpener-f0c80b795ea0c9c5ec8ad26a6060e403-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/family1 2023-10-27 23:59:03,492 INFO [StoreOpener-f0c80b795ea0c9c5ec8ad26a6060e403-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f0c80b795ea0c9c5ec8ad26a6060e403 columnFamilyName family1 2023-10-27 23:59:03,493 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=31, resume processing ppid=28 2023-10-27 23:59:03,493 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=31, ppid=28, state=SUCCESS; OpenRegionProcedure db3340090786ac1f09ea9c45bcb224a0, server=jenkins-hbase16.apache.org,43469,1698451096272 in 191 msec 2023-10-27 23:59:03,494 INFO [StoreOpener-f0c80b795ea0c9c5ec8ad26a6060e403-1] regionserver.HStore(310): Store=f0c80b795ea0c9c5ec8ad26a6060e403/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-27 23:59:03,494 INFO [StoreOpener-f0c80b795ea0c9c5ec8ad26a6060e403-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region f0c80b795ea0c9c5ec8ad26a6060e403 2023-10-27 23:59:03,494 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=db3340090786ac1f09ea9c45bcb224a0, ASSIGN in 357 msec 2023-10-27 23:59:03,496 DEBUG [StoreOpener-f0c80b795ea0c9c5ec8ad26a6060e403-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/family2 2023-10-27 23:59:03,496 DEBUG [StoreOpener-f0c80b795ea0c9c5ec8ad26a6060e403-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/family2 2023-10-27 23:59:03,497 INFO [StoreOpener-f0c80b795ea0c9c5ec8ad26a6060e403-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f0c80b795ea0c9c5ec8ad26a6060e403 columnFamilyName family2 2023-10-27 23:59:03,498 INFO [StoreOpener-f0c80b795ea0c9c5ec8ad26a6060e403-1] regionserver.HStore(310): Store=f0c80b795ea0c9c5ec8ad26a6060e403/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-27 23:59:03,499 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403 2023-10-27 23:59:03,499 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403 2023-10-27 23:59:03,501 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-10-27 23:59:03,503 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for f0c80b795ea0c9c5ec8ad26a6060e403 2023-10-27 23:59:03,504 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-10-27 23:59:03,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened f0c80b795ea0c9c5ec8ad26a6060e403; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59029946, jitterRate=-0.12038525938987732}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-10-27 23:59:03,505 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for f0c80b795ea0c9c5ec8ad26a6060e403: 2023-10-27 23:59:03,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403., pid=32, masterSystemTime=1698451143453 2023-10-27 23:59:03,507 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403. 2023-10-27 23:59:03,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403. 2023-10-27 23:59:03,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open testScannerOnBulkLoadRefHFiles,1,1698451142635.a91badaa245ed444aa843d100a54dfca. 2023-10-27 23:59:03,508 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a91badaa245ed444aa843d100a54dfca, NAME => 'testScannerOnBulkLoadRefHFiles,1,1698451142635.a91badaa245ed444aa843d100a54dfca.', STARTKEY => '1', ENDKEY => '2'} 2023-10-27 23:59:03,508 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=f0c80b795ea0c9c5ec8ad26a6060e403, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:03,508 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles a91badaa245ed444aa843d100a54dfca 2023-10-27 23:59:03,508 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1698451143508"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451143508"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451143508"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451143508"}]},"ts":"1698451143508"} 2023-10-27 23:59:03,508 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1698451142635.a91badaa245ed444aa843d100a54dfca.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:59:03,508 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for a91badaa245ed444aa843d100a54dfca 2023-10-27 23:59:03,508 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for a91badaa245ed444aa843d100a54dfca 2023-10-27 23:59:03,510 INFO [StoreOpener-a91badaa245ed444aa843d100a54dfca-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a91badaa245ed444aa843d100a54dfca 2023-10-27 23:59:03,512 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=32, resume processing ppid=30 2023-10-27 23:59:03,512 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=32, ppid=30, state=SUCCESS; OpenRegionProcedure f0c80b795ea0c9c5ec8ad26a6060e403, server=jenkins-hbase16.apache.org,43469,1698451096272 in 209 msec 2023-10-27 23:59:03,512 DEBUG [StoreOpener-a91badaa245ed444aa843d100a54dfca-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/a91badaa245ed444aa843d100a54dfca/family1 2023-10-27 23:59:03,512 DEBUG [StoreOpener-a91badaa245ed444aa843d100a54dfca-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/a91badaa245ed444aa843d100a54dfca/family1 2023-10-27 23:59:03,513 INFO [StoreOpener-a91badaa245ed444aa843d100a54dfca-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a91badaa245ed444aa843d100a54dfca columnFamilyName family1 2023-10-27 23:59:03,514 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=f0c80b795ea0c9c5ec8ad26a6060e403, ASSIGN in 376 msec 2023-10-27 23:59:03,514 INFO [StoreOpener-a91badaa245ed444aa843d100a54dfca-1] regionserver.HStore(310): Store=a91badaa245ed444aa843d100a54dfca/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-27 23:59:03,515 INFO [StoreOpener-a91badaa245ed444aa843d100a54dfca-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region a91badaa245ed444aa843d100a54dfca 2023-10-27 23:59:03,516 DEBUG [StoreOpener-a91badaa245ed444aa843d100a54dfca-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/a91badaa245ed444aa843d100a54dfca/family2 2023-10-27 23:59:03,516 DEBUG [StoreOpener-a91badaa245ed444aa843d100a54dfca-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/a91badaa245ed444aa843d100a54dfca/family2 2023-10-27 23:59:03,517 INFO [StoreOpener-a91badaa245ed444aa843d100a54dfca-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a91badaa245ed444aa843d100a54dfca columnFamilyName family2 2023-10-27 23:59:03,518 INFO [StoreOpener-a91badaa245ed444aa843d100a54dfca-1] regionserver.HStore(310): Store=a91badaa245ed444aa843d100a54dfca/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-27 23:59:03,519 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/a91badaa245ed444aa843d100a54dfca 2023-10-27 23:59:03,519 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/a91badaa245ed444aa843d100a54dfca 2023-10-27 23:59:03,521 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-10-27 23:59:03,522 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for a91badaa245ed444aa843d100a54dfca 2023-10-27 23:59:03,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/a91badaa245ed444aa843d100a54dfca/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-10-27 23:59:03,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened a91badaa245ed444aa843d100a54dfca; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66133657, jitterRate=-0.014531716704368591}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-10-27 23:59:03,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for a91badaa245ed444aa843d100a54dfca: 2023-10-27 23:59:03,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1698451142635.a91badaa245ed444aa843d100a54dfca., pid=33, masterSystemTime=1698451143453 2023-10-27 23:59:03,527 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1698451142635.a91badaa245ed444aa843d100a54dfca. 2023-10-27 23:59:03,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened testScannerOnBulkLoadRefHFiles,1,1698451142635.a91badaa245ed444aa843d100a54dfca. 2023-10-27 23:59:03,528 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=a91badaa245ed444aa843d100a54dfca, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:03,528 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1698451142635.a91badaa245ed444aa843d100a54dfca.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1698451143528"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451143528"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451143528"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451143528"}]},"ts":"1698451143528"} 2023-10-27 23:59:03,532 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=33, resume processing ppid=29 2023-10-27 23:59:03,532 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=33, ppid=29, state=SUCCESS; OpenRegionProcedure a91badaa245ed444aa843d100a54dfca, server=jenkins-hbase16.apache.org,43469,1698451096272 in 228 msec 2023-10-27 23:59:03,534 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=29, resume processing ppid=27 2023-10-27 23:59:03,534 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=a91badaa245ed444aa843d100a54dfca, ASSIGN in 396 msec 2023-10-27 23:59:03,534 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-10-27 23:59:03,535 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451143534"}]},"ts":"1698451143534"} 2023-10-27 23:59:03,536 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-10-27 23:59:03,570 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-10-27 23:59:03,572 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 935 msec 2023-10-27 23:59:03,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=27 2023-10-27 23:59:03,746 INFO [Listener at localhost/40295] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-10-27 23:59:03,753 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:03,755 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:03,756 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-10-27 23:59:03,756 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:03,756 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-10-27 23:59:03,756 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-10-27 23:59:03,756 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:03,756 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:03,760 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:03,760 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:03,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:03,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:03,760 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:03,761 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:03,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:03,761 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:03,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:03,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:03,761 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:03,762 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:03,762 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-10-27 23:59:03,762 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:03,762 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-10-27 23:59:03,762 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1698451142635.a91badaa245ed444aa843d100a54dfca. 2023-10-27 23:59:03,762 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1698451142635.a91badaa245ed444aa843d100a54dfca. started... 2023-10-27 23:59:03,762 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-10-27 23:59:03,762 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:03,762 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403. 2023-10-27 23:59:03,763 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a91badaa245ed444aa843d100a54dfca: 2023-10-27 23:59:03,762 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1698451142635.db3340090786ac1f09ea9c45bcb224a0. 2023-10-27 23:59:03,763 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1698451142635.a91badaa245ed444aa843d100a54dfca. 2023-10-27 23:59:03,763 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403. started... 2023-10-27 23:59:03,763 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-10-27 23:59:03,763 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1698451142635.db3340090786ac1f09ea9c45bcb224a0. started... 2023-10-27 23:59:03,764 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing f0c80b795ea0c9c5ec8ad26a6060e403 2/2 column families, dataSize=48 B heapSize=632 B 2023-10-27 23:59:03,764 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for db3340090786ac1f09ea9c45bcb224a0: 2023-10-27 23:59:03,765 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1698451142635.db3340090786ac1f09ea9c45bcb224a0. 2023-10-27 23:59:03,765 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-10-27 23:59:04,187 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231027818b7ca483874d1b9bfd756d97dcb554 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20231027818b7ca483874d1b9bfd756d97dcb554 2023-10-27 23:59:04,193 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/.tmp/family1/1188a3539a94466d89f2f684a5248880 2023-10-27 23:59:04,199 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/.tmp/family1/1188a3539a94466d89f2f684a5248880 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/family1/1188a3539a94466d89f2f684a5248880 2023-10-27 23:59:04,204 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/family1/1188a3539a94466d89f2f684a5248880, entries=1, sequenceid=5, filesize=4.9 K 2023-10-27 23:59:04,205 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for f0c80b795ea0c9c5ec8ad26a6060e403 in 441ms, sequenceid=5, compaction requested=false 2023-10-27 23:59:04,206 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-10-27 23:59:04,206 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for f0c80b795ea0c9c5ec8ad26a6060e403: 2023-10-27 23:59:04,207 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403. 2023-10-27 23:59:04,207 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-10-27 23:59:04,207 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-10-27 23:59:04,207 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:04,207 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-10-27 23:59:04,207 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-10-27 23:59:04,235 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,235 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:04,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:04,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:04,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:04,236 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,236 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:04,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:04,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:04,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:04,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:04,237 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-10-27 23:59:04,237 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@60b1bd4b[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:04,237 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-10-27 23:59:04,237 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,244 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,244 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:04,244 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-10-27 23:59:04,244 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-10-27 23:59:04,244 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:04,244 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:04,252 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-10-27 23:59:04,252 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:04,252 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:04,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:04,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:04,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:04,252 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-10-27 23:59:04,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:04,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:04,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:04,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:04,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:04,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:04,255 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-10-27 23:59:04,255 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-10-27 23:59:04,255 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:04,255 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:04,255 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,255 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:04,260 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,260 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,260 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:04,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:04,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:04,261 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:04,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:04,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:04,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:04,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:04,262 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:04,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:04,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:04,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:04,290 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:04,291 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:04,291 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-10-27 23:59:04,291 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:04,291 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:04,291 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:04,291 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-10-27 23:59:04,291 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:04,291 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:04,291 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:04,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:04,291 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,291 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:04,291 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,291 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,291 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:04,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:04,391 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:04,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:04,400 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:04,403 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-10-27 23:59:04,403 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,403 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-10-27 23:59:04,403 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:04,404 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-10-27 23:59:04,404 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-10-27 23:59:04,404 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,404 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,417 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:04,417 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:04,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:04,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:04,418 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:04,418 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:04,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,418 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:04,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,420 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,420 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-10-27 23:59:04,420 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:04,420 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-10-27 23:59:04,420 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:04,420 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-10-27 23:59:04,421 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1698451142635.a91badaa245ed444aa843d100a54dfca. 2023-10-27 23:59:04,421 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1698451142635.a91badaa245ed444aa843d100a54dfca. started... 2023-10-27 23:59:04,421 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403. 2023-10-27 23:59:04,421 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a91badaa245ed444aa843d100a54dfca: 2023-10-27 23:59:04,421 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1698451142635.db3340090786ac1f09ea9c45bcb224a0. 2023-10-27 23:59:04,421 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1698451142635.db3340090786ac1f09ea9c45bcb224a0. started... 2023-10-27 23:59:04,421 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-10-27 23:59:04,421 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for db3340090786ac1f09ea9c45bcb224a0: 2023-10-27 23:59:04,421 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1698451142635.db3340090786ac1f09ea9c45bcb224a0. 2023-10-27 23:59:04,421 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:04,421 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1698451142635.a91badaa245ed444aa843d100a54dfca. 2023-10-27 23:59:04,421 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403. started... 2023-10-27 23:59:04,422 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-10-27 23:59:04,422 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing f0c80b795ea0c9c5ec8ad26a6060e403 2/2 column families, dataSize=90 B heapSize=752 B 2023-10-27 23:59:04,422 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-10-27 23:59:04,849 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202310278f89f75cc3494bc99b7095b5179feb9c to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202310278f89f75cc3494bc99b7095b5179feb9c 2023-10-27 23:59:05,257 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/.tmp/family1/1b6e099f2a024897977d29eda17b2873 2023-10-27 23:59:05,272 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/.tmp/family1/1b6e099f2a024897977d29eda17b2873 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/family1/1b6e099f2a024897977d29eda17b2873 2023-10-27 23:59:05,279 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/family1/1b6e099f2a024897977d29eda17b2873, entries=2, sequenceid=9, filesize=5.0 K 2023-10-27 23:59:05,280 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for f0c80b795ea0c9c5ec8ad26a6060e403 in 858ms, sequenceid=9, compaction requested=false 2023-10-27 23:59:05,281 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for f0c80b795ea0c9c5ec8ad26a6060e403: 2023-10-27 23:59:05,281 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403. 2023-10-27 23:59:05,281 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-10-27 23:59:05,282 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-10-27 23:59:05,282 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:05,282 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-10-27 23:59:05,282 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-10-27 23:59:05,310 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:05,310 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:05,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:05,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:05,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:05,311 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:05,312 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:05,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:05,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:05,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:05,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:05,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:05,316 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-10-27 23:59:05,316 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4de6f418[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:05,316 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-10-27 23:59:05,316 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:05,326 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:05,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:05,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:05,326 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-10-27 23:59:05,326 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-10-27 23:59:05,326 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:05,327 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:05,327 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:05,336 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-10-27 23:59:05,336 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:05,336 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-10-27 23:59:05,336 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:05,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:05,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:05,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:05,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:05,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:05,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:05,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:05,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:05,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:05,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:05,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:05,345 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-10-27 23:59:05,345 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-10-27 23:59:05,346 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:05,346 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:05,346 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:05,346 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:05,352 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:05,352 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:05,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:05,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:05,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:05,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:05,352 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:05,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:05,352 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:05,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:05,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:05,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:05,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:05,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:05,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:05,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:05,353 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:05,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:05,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:05,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:05,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:05,360 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:05,360 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:05,360 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:05,360 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:05,360 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-10-27 23:59:05,360 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:05,360 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:05,360 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:05,360 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-10-27 23:59:05,360 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:05,360 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:05,361 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:05,361 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:05,361 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:05,361 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:05,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:05,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:05,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:05,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:05,461 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:05,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:05,477 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36413] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-10-27 23:59:05,479 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] procedure2.ProcedureExecutor(1028): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-10-27 23:59:05,480 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-10-27 23:59:05,481 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-27 23:59:05,481 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698451745481 2023-10-27 23:59:05,482 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-10-27 23:59:05,483 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-10-27 23:59:05,483 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-10-27 23:59:05,485 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] zookeeper.ReadOnlyZKClient(139): Connect 0x2470ae6f to 127.0.0.1:57369 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-10-27 23:59:05,494 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7267183b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-27 23:59:05,494 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20231027 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-10-27 23:59:06,341 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023102710b1f73af48841c39d1bf7c494f54987 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023102710b1f73af48841c39d1bf7c494f54987 2023-10-27 23:59:06,343 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231027 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-10-27 23:59:06,363 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-10-27 23:59:06,364 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:51350, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-10-27 23:59:06,371 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-10-27 23:59:06,373 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:40724, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-10-27 23:59:06,384 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20231027/family1/76b068b30b084a31aa7f992b1c0a7d85 first=Optional[k0] last=Optional[k1] 2023-10-27 23:59:06,385 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20231027/family1/76b068b30b084a31aa7f992b1c0a7d85}] 2023-10-27 23:59:06,394 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20231027/family1/76b068b30b084a31aa7f992b1c0a7d85 for inclusion in f0c80b795ea0c9c5ec8ad26a6060e403/family1 2023-10-27 23:59:06,398 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-10-27 23:59:06,398 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(777): Region bounds: first=2 last= 2023-10-27 23:59:06,398 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for f0c80b795ea0c9c5ec8ad26a6060e403: 2023-10-27 23:59:06,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20231027/family1/76b068b30b084a31aa7f992b1c0a7d85 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testScannerOnBulkLoadRefHFiles__n2n2kufpljb5ghhv0d005mjsmj0gebldg63s4pj6it9lfsaksu2ggk2ol3t5rm23/family1/76b068b30b084a31aa7f992b1c0a7d85 2023-10-27 23:59:06,403 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testScannerOnBulkLoadRefHFiles__n2n2kufpljb5ghhv0d005mjsmj0gebldg63s4pj6it9lfsaksu2ggk2ol3t5rm23/family1/76b068b30b084a31aa7f992b1c0a7d85 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/family1/1307920b4d5744a9a415a99f66cba547_SeqId_12_ 2023-10-27 23:59:06,404 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testScannerOnBulkLoadRefHFiles__n2n2kufpljb5ghhv0d005mjsmj0gebldg63s4pj6it9lfsaksu2ggk2ol3t5rm23/family1/76b068b30b084a31aa7f992b1c0a7d85 into f0c80b795ea0c9c5ec8ad26a6060e403/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/family1/1307920b4d5744a9a415a99f66cba547_SeqId_12_ - updating store file list. 2023-10-27 23:59:06,410 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/family1/1307920b4d5744a9a415a99f66cba547_SeqId_12_ into f0c80b795ea0c9c5ec8ad26a6060e403/family1 2023-10-27 23:59:06,410 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testScannerOnBulkLoadRefHFiles__n2n2kufpljb5ghhv0d005mjsmj0gebldg63s4pj6it9lfsaksu2ggk2ol3t5rm23/family1/76b068b30b084a31aa7f992b1c0a7d85 into f0c80b795ea0c9c5ec8ad26a6060e403/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/family1/1307920b4d5744a9a415a99f66cba547_SeqId_12_) 2023-10-27 23:59:06,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testScannerOnBulkLoadRefHFiles__n2n2kufpljb5ghhv0d005mjsmj0gebldg63s4pj6it9lfsaksu2ggk2ol3t5rm23/family1/76b068b30b084a31aa7f992b1c0a7d85 2023-10-27 23:59:06,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-10-27 23:59:06,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-10-27 23:59:06,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(1912): f0c80b795ea0c9c5ec8ad26a6060e403/family1 is initiating minor compaction (all files) 2023-10-27 23:59:06,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@5806123f; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-27 23:59:06,413 INFO [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.HRegion(2259): Starting compaction of f0c80b795ea0c9c5ec8ad26a6060e403/family1 in testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403. 2023-10-27 23:59:06,413 INFO [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/family1/1188a3539a94466d89f2f684a5248880, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/family1/1b6e099f2a024897977d29eda17b2873, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/family1/1307920b4d5744a9a415a99f66cba547_SeqId_12_] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/.tmp, totalSize=15.1 K 2023-10-27 23:59:06,413 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] compactions.Compactor(207): Compacting 1188a3539a94466d89f2f684a5248880, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698451143746 2023-10-27 23:59:06,414 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] compactions.Compactor(207): Compacting 1b6e099f2a024897977d29eda17b2873, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698451143746 2023-10-27 23:59:06,414 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] compactions.Compactor(207): Compacting 1307920b4d5744a9a415a99f66cba547_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698451143746 2023-10-27 23:59:06,415 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231027 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-10-27 23:59:06,416 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:59:06,418 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20231027818b7ca483874d1b9bfd756d97dcb554 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20231027818b7ca483874d1b9bfd756d97dcb554 2023-10-27 23:59:06,440 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202310278f89f75cc3494bc99b7095b5179feb9c to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202310278f89f75cc3494bc99b7095b5179feb9c 2023-10-27 23:59:06,442 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-10-27 23:59:06,442 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-10-27 23:59:06,443 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x551ac9e9 to 127.0.0.1:57369 2023-10-27 23:59:06,442 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20231027' 2023-10-27 23:59:06,443 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-10-27 23:59:06,444 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20231027 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-10-27 23:59:06,444 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-10-27 23:59:06,444 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-27 23:59:06,444 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-10-27 23:59:06,445 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x065c1da2 to 127.0.0.1:57369 2023-10-27 23:59:06,445 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-10-27 23:59:06,445 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-10-27 23:59:06,451 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-10-27 23:59:06,451 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5073ebf1 to 127.0.0.1:57369 2023-10-27 23:59:06,451 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-10-27 23:59:06,451 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-10-27 23:59:06,452 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x27ac06ae to 127.0.0.1:57369 2023-10-27 23:59:06,452 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-10-27 23:59:06,453 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 967 msec 2023-10-27 23:59:06,456 INFO [RS:0;jenkins-hbase16:43469-longCompactions-0] throttle.PressureAwareThroughputController(145): f0c80b795ea0c9c5ec8ad26a6060e403#family1#compaction#38 average throughput is 0.05 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-10-27 23:59:06,481 INFO [Listener at localhost/40295] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=268 (was 263) - Thread LEAK? -, OpenFileDescriptor=618 (was 627), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=250 (was 229) - SystemLoadAverage LEAK? -, ProcessCount=165 (was 167), AvailableMemoryMB=6055 (was 4888) - AvailableMemoryMB LEAK? - 2023-10-27 23:59:06,492 INFO [Listener at localhost/40295] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=268, OpenFileDescriptor=618, MaxFileDescriptor=60000, SystemLoadAverage=250, ProcessCount=165, AvailableMemoryMB=6054 2023-10-27 23:59:06,494 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.HMaster$4(2112): Client=jenkins//188.40.62.62 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-10-27 23:59:06,495 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure2.ProcedureExecutor(1028): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:06,496 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-10-27 23:59:06,497 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(700): Client=jenkins//188.40.62.62 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-10-27 23:59:06,497 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-10-27 23:59:06,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-10-27 23:59:06,499 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e 2023-10-27 23:59:06,499 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e empty. 2023-10-27 23:59:06,500 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e 2023-10-27 23:59:06,500 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-10-27 23:59:06,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-10-27 23:59:06,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-10-27 23:59:06,911 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-10-27 23:59:06,914 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 738e5e9e8c019e543eb40425cbae960e, NAME => 'testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp 2023-10-27 23:59:07,105 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-10-27 23:59:07,283 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/.tmp/family1/74efbb8be81d45879797d2a28bbfcda5 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/family1/74efbb8be81d45879797d2a28bbfcda5 2023-10-27 23:59:07,289 INFO [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in f0c80b795ea0c9c5ec8ad26a6060e403/family1 of f0c80b795ea0c9c5ec8ad26a6060e403 into 74efbb8be81d45879797d2a28bbfcda5(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-10-27 23:59:07,289 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for f0c80b795ea0c9c5ec8ad26a6060e403: 2023-10-27 23:59:07,290 INFO [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1698451146413; duration=0sec 2023-10-27 23:59:07,290 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-27 23:59:07,326 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:59:07,326 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing 738e5e9e8c019e543eb40425cbae960e, disabling compactions & flushes 2023-10-27 23:59:07,326 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:07,326 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:07,326 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. after waiting 0 ms 2023-10-27 23:59:07,326 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:07,326 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:07,326 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 738e5e9e8c019e543eb40425cbae960e: 2023-10-27 23:59:07,331 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-10-27 23:59:07,333 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1698451147333"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451147333"}]},"ts":"1698451147333"} 2023-10-27 23:59:07,336 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-10-27 23:59:07,340 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/family1/1188a3539a94466d89f2f684a5248880, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/family1/1b6e099f2a024897977d29eda17b2873, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/family1/1307920b4d5744a9a415a99f66cba547_SeqId_12_] to archive 2023-10-27 23:59:07,340 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-10-27 23:59:07,340 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451147340"}]},"ts":"1698451147340"} 2023-10-27 23:59:07,341 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:59:07,342 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-10-27 23:59:07,343 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/family1/1188a3539a94466d89f2f684a5248880 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/family1/1188a3539a94466d89f2f684a5248880 2023-10-27 23:59:07,344 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/family1/1b6e099f2a024897977d29eda17b2873 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/family1/1b6e099f2a024897977d29eda17b2873 2023-10-27 23:59:07,346 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/family1/1307920b4d5744a9a415a99f66cba547_SeqId_12_ to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/family1/1307920b4d5744a9a415a99f66cba547_SeqId_12_ 2023-10-27 23:59:07,385 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=738e5e9e8c019e543eb40425cbae960e, ASSIGN}] 2023-10-27 23:59:07,388 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=738e5e9e8c019e543eb40425cbae960e, ASSIGN 2023-10-27 23:59:07,389 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=738e5e9e8c019e543eb40425cbae960e, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,43469,1698451096272; forceNewPlan=false, retain=false 2023-10-27 23:59:07,541 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=738e5e9e8c019e543eb40425cbae960e, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:07,541 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1698451147541"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451147541"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451147541"}]},"ts":"1698451147541"} 2023-10-27 23:59:07,545 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure 738e5e9e8c019e543eb40425cbae960e, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-27 23:59:07,607 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-10-27 23:59:07,699 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:07,705 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:07,706 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 738e5e9e8c019e543eb40425cbae960e, NAME => 'testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e.', STARTKEY => '', ENDKEY => ''} 2023-10-27 23:59:07,706 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy 738e5e9e8c019e543eb40425cbae960e 2023-10-27 23:59:07,706 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:59:07,706 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for 738e5e9e8c019e543eb40425cbae960e 2023-10-27 23:59:07,706 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for 738e5e9e8c019e543eb40425cbae960e 2023-10-27 23:59:07,709 INFO [StoreOpener-738e5e9e8c019e543eb40425cbae960e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 738e5e9e8c019e543eb40425cbae960e 2023-10-27 23:59:07,713 DEBUG [StoreOpener-738e5e9e8c019e543eb40425cbae960e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1 2023-10-27 23:59:07,713 DEBUG [StoreOpener-738e5e9e8c019e543eb40425cbae960e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1 2023-10-27 23:59:07,714 INFO [StoreOpener-738e5e9e8c019e543eb40425cbae960e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 738e5e9e8c019e543eb40425cbae960e columnFamilyName family1 2023-10-27 23:59:07,716 INFO [StoreOpener-738e5e9e8c019e543eb40425cbae960e-1] regionserver.HStore(310): Store=738e5e9e8c019e543eb40425cbae960e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-27 23:59:07,717 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e 2023-10-27 23:59:07,717 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e 2023-10-27 23:59:07,720 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for 738e5e9e8c019e543eb40425cbae960e 2023-10-27 23:59:07,722 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-10-27 23:59:07,723 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened 738e5e9e8c019e543eb40425cbae960e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68188541, jitterRate=0.016088441014289856}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-10-27 23:59:07,723 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for 738e5e9e8c019e543eb40425cbae960e: 2023-10-27 23:59:07,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e., pid=37, masterSystemTime=1698451147699 2023-10-27 23:59:07,725 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:07,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:07,725 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=738e5e9e8c019e543eb40425cbae960e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:07,726 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1698451147725"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451147725"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451147725"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451147725"}]},"ts":"1698451147725"} 2023-10-27 23:59:07,729 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=37, resume processing ppid=36 2023-10-27 23:59:07,729 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure 738e5e9e8c019e543eb40425cbae960e, server=jenkins-hbase16.apache.org,43469,1698451096272 in 182 msec 2023-10-27 23:59:07,730 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=36, resume processing ppid=35 2023-10-27 23:59:07,730 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=738e5e9e8c019e543eb40425cbae960e, ASSIGN in 344 msec 2023-10-27 23:59:07,731 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-10-27 23:59:07,731 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451147731"}]},"ts":"1698451147731"} 2023-10-27 23:59:07,732 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-10-27 23:59:07,743 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-10-27 23:59:07,744 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 1.2490 sec 2023-10-27 23:59:08,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-10-27 23:59:08,609 INFO [Listener at localhost/40295] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-10-27 23:59:08,637 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:08,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:08,639 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-10-27 23:59:08,639 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:08,639 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-10-27 23:59:08,639 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-10-27 23:59:08,640 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:08,640 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:08,668 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:08,668 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:08,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:08,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:08,669 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:08,669 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:08,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:08,670 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:08,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:08,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:08,670 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:08,670 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:08,670 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-10-27 23:59:08,670 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:08,670 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-10-27 23:59:08,670 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:08,671 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:08,671 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:08,671 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. started... 2023-10-27 23:59:08,671 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 738e5e9e8c019e543eb40425cbae960e 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-10-27 23:59:08,992 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-10-27 23:59:09,094 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511309a6031f1b0944ab5b0eb1e9b384a6e09 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511309a6031f1b0944ab5b0eb1e9b384a6e09 2023-10-27 23:59:09,500 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/.tmp/family1/ecf144b8ce904147b1e90baed078915a 2023-10-27 23:59:09,518 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/.tmp/family1/ecf144b8ce904147b1e90baed078915a as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/ecf144b8ce904147b1e90baed078915a 2023-10-27 23:59:09,525 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/ecf144b8ce904147b1e90baed078915a, entries=1000, sequenceid=5, filesize=165.1 K 2023-10-27 23:59:09,526 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 738e5e9e8c019e543eb40425cbae960e in 855ms, sequenceid=5, compaction requested=false 2023-10-27 23:59:09,526 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-10-27 23:59:09,527 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 738e5e9e8c019e543eb40425cbae960e: 2023-10-27 23:59:09,527 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:09,527 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:09,527 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:09,527 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:09,527 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-10-27 23:59:09,527 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-10-27 23:59:09,552 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:09,552 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:09,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:09,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:09,553 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,553 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:09,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:09,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:09,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:09,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:09,556 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-10-27 23:59:09,556 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2527830c[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:09,556 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-10-27 23:59:09,556 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,567 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,568 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-10-27 23:59:09,568 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-10-27 23:59:09,568 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:09,568 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:09,568 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:09,577 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:09,577 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-10-27 23:59:09,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:09,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:09,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:09,577 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:09,578 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-10-27 23:59:09,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:09,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:09,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:09,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:09,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:09,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:09,586 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:59:09,586 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-10-27 23:59:09,586 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:09,587 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:09,587 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,587 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:09,594 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,594 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,594 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:09,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:09,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:09,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:09,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:09,595 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:09,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:09,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:09,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,598 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:09,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:09,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:09,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:09,610 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:09,610 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:09,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:09,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:09,610 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:09,610 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-10-27 23:59:09,611 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:09,610 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:09,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:09,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:09,611 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:09,611 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:09,610 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,611 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,612 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:09,612 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,611 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-10-27 23:59:09,612 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,612 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,712 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:09,714 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:09,724 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:09,725 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-10-27 23:59:09,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,726 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-10-27 23:59:09,726 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:09,726 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-10-27 23:59:09,726 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-10-27 23:59:09,726 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,726 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,735 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:09,735 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:09,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:09,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:09,736 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:09,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,736 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:09,737 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:09,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,738 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,738 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-10-27 23:59:09,738 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:09,738 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-10-27 23:59:09,738 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:09,738 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-10-27 23:59:09,739 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:09,739 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:09,739 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:09,739 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. started... 2023-10-27 23:59:09,739 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 738e5e9e8c019e543eb40425cbae960e 1/1 column families, dataSize=74 B heapSize=408 B 2023-10-27 23:59:10,165 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511281fb1c76f7803498db525ea32b7831675 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511281fb1c76f7803498db525ea32b7831675 2023-10-27 23:59:10,577 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/.tmp/family1/5eeb20f4fd1d4cb78631e312bae38eb9 2023-10-27 23:59:10,588 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/.tmp/family1/5eeb20f4fd1d4cb78631e312bae38eb9 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/5eeb20f4fd1d4cb78631e312bae38eb9 2023-10-27 23:59:10,594 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/5eeb20f4fd1d4cb78631e312bae38eb9, entries=1, sequenceid=9, filesize=5.0 K 2023-10-27 23:59:10,595 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 738e5e9e8c019e543eb40425cbae960e in 856ms, sequenceid=9, compaction requested=false 2023-10-27 23:59:10,595 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 738e5e9e8c019e543eb40425cbae960e: 2023-10-27 23:59:10,595 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:10,595 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:10,596 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:10,596 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:10,596 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-10-27 23:59:10,596 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-10-27 23:59:10,644 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:10,644 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:10,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:10,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:10,645 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,645 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:10,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:10,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:10,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:10,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:10,650 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-10-27 23:59:10,650 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b7d2aa7[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:10,650 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-10-27 23:59:10,650 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,660 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,661 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-10-27 23:59:10,661 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-10-27 23:59:10,661 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:10,661 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:10,661 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:10,669 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-10-27 23:59:10,669 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:10,669 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-10-27 23:59:10,669 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:10,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:10,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:10,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:10,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:10,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:10,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:10,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:10,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:10,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:10,680 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:59:10,680 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-10-27 23:59:10,680 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:10,680 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:10,681 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,681 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:10,693 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,693 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:10,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:10,693 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:10,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:10,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:10,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:10,695 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:10,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:10,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,698 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:10,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:10,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:10,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:10,710 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:10,710 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:10,710 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:10,711 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-10-27 23:59:10,710 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:10,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:10,711 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-10-27 23:59:10,711 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:10,710 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:10,710 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,712 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:10,712 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,712 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:10,712 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:10,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:10,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:10,712 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,713 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,812 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:10,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:10,824 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:10,827 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-10-27 23:59:10,827 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,827 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-10-27 23:59:10,827 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:10,827 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-10-27 23:59:10,827 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-10-27 23:59:10,828 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,828 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,835 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:10,835 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:10,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:10,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:10,836 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:10,836 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:10,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,837 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:10,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,839 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,839 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-10-27 23:59:10,839 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:10,839 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-10-27 23:59:10,839 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:10,840 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-10-27 23:59:10,840 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:10,840 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:10,840 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:10,840 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. started... 2023-10-27 23:59:10,840 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 738e5e9e8c019e543eb40425cbae960e 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:59:11,267 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201c538a62a67434790a22f707c0ccaca5f to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201c538a62a67434790a22f707c0ccaca5f 2023-10-27 23:59:11,677 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/.tmp/family1/028cfadc4b6841359f04263a0020e571 2023-10-27 23:59:11,687 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/.tmp/family1/028cfadc4b6841359f04263a0020e571 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/028cfadc4b6841359f04263a0020e571 2023-10-27 23:59:11,693 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/028cfadc4b6841359f04263a0020e571, entries=1, sequenceid=13, filesize=4.9 K 2023-10-27 23:59:11,693 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 738e5e9e8c019e543eb40425cbae960e in 853ms, sequenceid=13, compaction requested=true 2023-10-27 23:59:11,694 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 738e5e9e8c019e543eb40425cbae960e: 2023-10-27 23:59:11,694 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:11,694 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:11,694 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:11,694 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:11,694 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-10-27 23:59:11,694 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-10-27 23:59:11,710 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:11,710 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:11,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:11,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:11,711 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,711 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:11,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:11,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:11,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:11,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:11,714 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-10-27 23:59:11,714 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@564b9366[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:11,714 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-10-27 23:59:11,714 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,718 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,719 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-10-27 23:59:11,719 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-10-27 23:59:11,719 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:11,719 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:11,719 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:11,727 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-10-27 23:59:11,727 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:11,727 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:11,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:11,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:11,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:11,727 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-10-27 23:59:11,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:11,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:11,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:11,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:11,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:11,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:11,734 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:59:11,734 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-10-27 23:59:11,735 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:11,735 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:11,735 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,735 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:11,743 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,743 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,743 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:11,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:11,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:11,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:11,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:11,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:11,744 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:11,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:11,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,745 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:11,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:11,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:11,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:11,751 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:11,751 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:11,752 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:11,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:11,752 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:11,752 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-10-27 23:59:11,752 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-10-27 23:59:11,752 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:11,752 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:11,752 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:11,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:11,752 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:11,752 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:11,752 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,752 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:11,752 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,753 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,853 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:11,854 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:11,862 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:11,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-10-27 23:59:11,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,865 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-10-27 23:59:11,865 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:11,865 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-10-27 23:59:11,865 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-10-27 23:59:11,866 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,866 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,877 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:11,877 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:11,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:11,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:11,878 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:11,878 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:11,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,880 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:11,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,881 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,881 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-10-27 23:59:11,882 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:11,882 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-10-27 23:59:11,882 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:11,882 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-10-27 23:59:11,882 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:11,882 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:11,883 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:11,883 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. started... 2023-10-27 23:59:11,883 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 738e5e9e8c019e543eb40425cbae960e 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:59:12,311 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205da9e7580e3c848648c68d283e1d940c0 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205da9e7580e3c848648c68d283e1d940c0 2023-10-27 23:59:12,721 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/.tmp/family1/38f5487b6808444bb4e6f6237e6c5023 2023-10-27 23:59:12,734 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/.tmp/family1/38f5487b6808444bb4e6f6237e6c5023 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/38f5487b6808444bb4e6f6237e6c5023 2023-10-27 23:59:12,742 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/38f5487b6808444bb4e6f6237e6c5023, entries=1, sequenceid=17, filesize=4.9 K 2023-10-27 23:59:12,743 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 738e5e9e8c019e543eb40425cbae960e in 860ms, sequenceid=17, compaction requested=true 2023-10-27 23:59:12,744 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 738e5e9e8c019e543eb40425cbae960e: 2023-10-27 23:59:12,744 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:12,744 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:12,745 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:12,745 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:12,745 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-10-27 23:59:12,745 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-10-27 23:59:12,777 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,777 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:12,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:12,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:12,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:12,777 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,777 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:12,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:12,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:12,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:12,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:12,779 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-10-27 23:59:12,779 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2cf7eb83[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:12,779 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-10-27 23:59:12,780 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,785 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,785 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-10-27 23:59:12,785 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-10-27 23:59:12,785 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:12,785 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:12,785 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:12,793 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-10-27 23:59:12,793 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:12,793 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:12,793 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-10-27 23:59:12,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:12,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:12,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:12,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:12,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:12,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:12,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:12,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:12,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:12,797 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:59:12,797 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-10-27 23:59:12,797 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:12,798 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:12,798 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,798 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:12,809 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,809 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:12,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:12,809 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:12,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:12,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:12,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:12,810 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:12,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:12,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,812 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:12,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:12,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:12,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:12,825 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:12,825 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,825 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,825 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:12,825 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,825 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-10-27 23:59:12,826 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:12,825 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:12,826 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,826 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,825 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:12,826 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:12,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:12,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:12,826 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:12,826 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:12,826 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-10-27 23:59:12,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:12,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:12,927 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:12,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:12,940 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:12,943 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-10-27 23:59:12,943 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,943 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-10-27 23:59:12,943 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:12,943 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-10-27 23:59:12,943 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-10-27 23:59:12,944 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,944 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,970 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:12,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:12,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:12,970 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:12,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,971 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:12,971 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:12,972 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:12,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,973 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,973 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-10-27 23:59:12,973 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:12,974 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-10-27 23:59:12,974 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:12,974 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-10-27 23:59:12,974 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:12,974 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:12,974 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:12,974 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. started... 2023-10-27 23:59:12,974 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 738e5e9e8c019e543eb40425cbae960e 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:59:13,403 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512289212aca782d845eb974ddb32bf736caf to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512289212aca782d845eb974ddb32bf736caf 2023-10-27 23:59:13,810 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/.tmp/family1/8c3e3a931e434f47ad3b0d7dec5a1cbb 2023-10-27 23:59:13,821 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/.tmp/family1/8c3e3a931e434f47ad3b0d7dec5a1cbb as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/8c3e3a931e434f47ad3b0d7dec5a1cbb 2023-10-27 23:59:13,827 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/8c3e3a931e434f47ad3b0d7dec5a1cbb, entries=1, sequenceid=21, filesize=4.9 K 2023-10-27 23:59:13,827 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 738e5e9e8c019e543eb40425cbae960e in 853ms, sequenceid=21, compaction requested=true 2023-10-27 23:59:13,828 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 738e5e9e8c019e543eb40425cbae960e: 2023-10-27 23:59:13,828 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:13,828 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:13,828 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:13,828 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:13,828 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-10-27 23:59:13,828 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-10-27 23:59:13,860 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:13,860 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:13,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:13,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:13,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:13,861 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:13,861 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:13,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:13,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:13,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:13,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:13,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:13,866 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-10-27 23:59:13,866 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@355122f2[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:13,866 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-10-27 23:59:13,867 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:13,877 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:13,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:13,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:13,877 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-10-27 23:59:13,877 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-10-27 23:59:13,877 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:13,877 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:13,877 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:13,885 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-10-27 23:59:13,885 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:13,885 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-10-27 23:59:13,885 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:13,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:13,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:13,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:13,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:13,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:13,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:13,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:13,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:13,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:13,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:13,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:13,891 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:59:13,891 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-10-27 23:59:13,891 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:13,892 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:13,892 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:13,892 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:13,902 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:13,902 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:13,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:13,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:13,902 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:13,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:13,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:13,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:13,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:13,903 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:13,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:13,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:13,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:13,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:13,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:13,906 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:13,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:13,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:13,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:13,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:13,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:13,918 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:13,918 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:13,919 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:13,919 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:13,919 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-10-27 23:59:13,919 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:13,919 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:13,919 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:13,919 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:13,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:13,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:13,920 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-10-27 23:59:13,919 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:13,920 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:13,920 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:13,919 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:13,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:13,920 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:13,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:14,020 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:14,022 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:14,031 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:14,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-10-27 23:59:14,034 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,034 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-10-27 23:59:14,034 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:14,034 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-10-27 23:59:14,034 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-10-27 23:59:14,034 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,035 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,043 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:14,043 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:14,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:14,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:14,044 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:14,044 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:14,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,044 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:14,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,045 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,045 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-10-27 23:59:14,046 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,046 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-10-27 23:59:14,046 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:14,046 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-10-27 23:59:14,046 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:14,046 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:14,046 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:14,046 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. started... 2023-10-27 23:59:14,046 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 738e5e9e8c019e543eb40425cbae960e 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:59:14,471 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231ee57de3d1ba541fe96d23bf9bbffe758 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231ee57de3d1ba541fe96d23bf9bbffe758 2023-10-27 23:59:14,879 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/.tmp/family1/228eb7051ac446ed8a653857dec9fe4f 2023-10-27 23:59:14,889 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/.tmp/family1/228eb7051ac446ed8a653857dec9fe4f as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/228eb7051ac446ed8a653857dec9fe4f 2023-10-27 23:59:14,897 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/228eb7051ac446ed8a653857dec9fe4f, entries=1, sequenceid=25, filesize=4.9 K 2023-10-27 23:59:14,898 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 738e5e9e8c019e543eb40425cbae960e in 852ms, sequenceid=25, compaction requested=true 2023-10-27 23:59:14,898 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 738e5e9e8c019e543eb40425cbae960e: 2023-10-27 23:59:14,898 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:14,898 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:14,898 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:14,898 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:14,898 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-10-27 23:59:14,898 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-10-27 23:59:14,927 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:14,927 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:14,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:14,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:14,927 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,928 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:14,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:14,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:14,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:14,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:14,932 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-10-27 23:59:14,932 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@67ab1eed[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:14,932 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-10-27 23:59:14,932 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,942 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,942 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-10-27 23:59:14,942 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-10-27 23:59:14,942 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:14,943 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:14,943 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:14,952 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-10-27 23:59:14,952 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:14,952 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-10-27 23:59:14,952 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:14,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:14,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:14,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:14,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:14,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:14,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:14,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:14,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:14,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:14,964 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:59:14,964 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-10-27 23:59:14,964 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:14,964 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:14,964 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,964 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:14,975 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,975 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,976 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:14,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:14,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:14,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:14,977 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:14,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:14,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:14,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:14,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,981 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:14,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:14,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:14,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:14,993 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:14,994 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,994 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,994 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-10-27 23:59:14,994 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:14,993 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:14,994 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:14,994 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:14,994 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:14,994 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:14,995 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-10-27 23:59:14,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:14,995 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,994 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:14,995 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:14,995 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:14,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:14,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:15,096 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:15,098 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:15,109 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:15,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-10-27 23:59:15,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:15,113 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-10-27 23:59:15,113 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:15,113 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-10-27 23:59:15,113 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-10-27 23:59:15,114 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:15,115 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:15,125 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:15,125 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:15,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:15,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:15,126 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:15,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:15,127 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:15,128 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:15,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:15,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:15,129 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:15,129 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-10-27 23:59:15,129 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:15,129 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-10-27 23:59:15,129 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:15,130 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-10-27 23:59:15,130 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:15,130 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:15,130 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:15,130 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. started... 2023-10-27 23:59:15,130 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 738e5e9e8c019e543eb40425cbae960e 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:59:15,560 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101ec3206c1de154fa18d80c8916bea8e0e to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101ec3206c1de154fa18d80c8916bea8e0e 2023-10-27 23:59:15,966 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/.tmp/family1/f9688645c0e748418ef4c9c559116781 2023-10-27 23:59:15,971 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/.tmp/family1/f9688645c0e748418ef4c9c559116781 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/f9688645c0e748418ef4c9c559116781 2023-10-27 23:59:15,976 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/f9688645c0e748418ef4c9c559116781, entries=1, sequenceid=29, filesize=4.9 K 2023-10-27 23:59:15,977 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 738e5e9e8c019e543eb40425cbae960e in 847ms, sequenceid=29, compaction requested=true 2023-10-27 23:59:15,977 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 738e5e9e8c019e543eb40425cbae960e: 2023-10-27 23:59:15,977 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:15,977 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:15,977 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:15,977 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:15,977 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-10-27 23:59:15,977 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-10-27 23:59:16,010 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,010 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:16,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:16,011 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,011 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:16,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:16,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:16,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:16,015 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-10-27 23:59:16,015 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6d7171dd[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:16,015 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-10-27 23:59:16,015 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,025 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,025 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-10-27 23:59:16,026 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-10-27 23:59:16,026 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:16,026 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,026 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:16,035 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-10-27 23:59:16,035 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,035 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:16,035 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-10-27 23:59:16,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:16,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:16,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:16,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:16,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:16,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:16,039 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:59:16,039 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-10-27 23:59:16,040 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:16,040 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:16,040 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,040 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:16,050 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,050 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:16,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:16,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,050 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:16,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:16,051 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:16,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:16,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:16,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,053 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:16,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,087 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,087 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,087 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,087 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,087 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:16,087 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-10-27 23:59:16,087 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:16,088 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:16,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:16,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:16,087 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-10-27 23:59:16,088 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:16,087 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,088 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,088 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:16,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:16,089 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:16,089 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:16,189 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:16,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:16,203 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:16,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-10-27 23:59:16,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,207 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-10-27 23:59:16,207 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:16,207 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-10-27 23:59:16,207 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-10-27 23:59:16,208 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,208 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,218 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,218 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:16,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:16,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:16,219 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,219 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:16,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,220 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:16,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,220 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,221 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-10-27 23:59:16,221 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,221 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-10-27 23:59:16,221 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:16,221 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-10-27 23:59:16,221 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:16,221 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:16,221 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:16,221 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. started... 2023-10-27 23:59:16,221 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 738e5e9e8c019e543eb40425cbae960e 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:59:16,642 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601033117933f8e4645ec9f3c2b93cd9d014b to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601033117933f8e4645ec9f3c2b93cd9d014b 2023-10-27 23:59:16,647 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/.tmp/family1/394c603a509042bdbd382e72c6998959 2023-10-27 23:59:16,652 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/.tmp/family1/394c603a509042bdbd382e72c6998959 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/394c603a509042bdbd382e72c6998959 2023-10-27 23:59:16,656 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/394c603a509042bdbd382e72c6998959, entries=1, sequenceid=33, filesize=4.9 K 2023-10-27 23:59:16,656 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 738e5e9e8c019e543eb40425cbae960e in 435ms, sequenceid=33, compaction requested=true 2023-10-27 23:59:16,657 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 738e5e9e8c019e543eb40425cbae960e: 2023-10-27 23:59:16,657 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:16,657 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:16,657 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:16,657 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,657 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-10-27 23:59:16,657 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-10-27 23:59:16,668 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,668 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:16,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:16,669 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:16,669 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:16,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:16,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:16,670 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-10-27 23:59:16,670 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7ec93403[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:16,670 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-10-27 23:59:16,670 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,676 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,677 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-10-27 23:59:16,677 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-10-27 23:59:16,677 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:16,677 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,677 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:16,685 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-10-27 23:59:16,685 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:16,685 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-10-27 23:59:16,685 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:16,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:16,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:16,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:16,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:16,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:16,689 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:59:16,689 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-10-27 23:59:16,690 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:16,690 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:16,690 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,690 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:16,700 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,700 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:16,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:16,700 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:16,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:16,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:16,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:16,701 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:16,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,705 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:16,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,718 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:16,718 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:16,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:16,719 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-10-27 23:59:16,719 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:16,718 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:16,719 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:16,719 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:16,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:16,720 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:16,720 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:16,720 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,720 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,721 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,721 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,721 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,820 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:16,822 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:16,827 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:16,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-10-27 23:59:16,829 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,829 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-10-27 23:59:16,829 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:16,829 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-10-27 23:59:16,829 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-10-27 23:59:16,829 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,830 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,835 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:16,835 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:16,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:16,836 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:16,836 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:16,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,837 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:16,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,837 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,838 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-10-27 23:59:16,838 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:16,838 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-10-27 23:59:16,838 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:16,838 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-10-27 23:59:16,838 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:16,838 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:16,838 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:16,838 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. started... 2023-10-27 23:59:16,839 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 738e5e9e8c019e543eb40425cbae960e 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:59:17,265 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090763e3d69b681d4a36be90c6f92f446c0f to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015090763e3d69b681d4a36be90c6f92f446c0f 2023-10-27 23:59:17,673 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/.tmp/family1/4fc5506c47ca4d63a24bd2911adc87fe 2023-10-27 23:59:17,683 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/.tmp/family1/4fc5506c47ca4d63a24bd2911adc87fe as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/4fc5506c47ca4d63a24bd2911adc87fe 2023-10-27 23:59:17,690 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/4fc5506c47ca4d63a24bd2911adc87fe, entries=1, sequenceid=37, filesize=4.9 K 2023-10-27 23:59:17,691 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 738e5e9e8c019e543eb40425cbae960e in 852ms, sequenceid=37, compaction requested=true 2023-10-27 23:59:17,692 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 738e5e9e8c019e543eb40425cbae960e: 2023-10-27 23:59:17,692 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:17,692 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:17,692 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:17,692 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:17,692 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-10-27 23:59:17,692 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-10-27 23:59:17,727 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:17,727 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:17,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:17,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:17,727 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,728 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:17,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:17,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:17,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:17,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:17,732 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-10-27 23:59:17,732 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4233df9a[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:17,732 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-10-27 23:59:17,732 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,743 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,744 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-10-27 23:59:17,744 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-10-27 23:59:17,744 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:17,744 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:17,745 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:17,752 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:17,752 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-10-27 23:59:17,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:17,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:17,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:17,752 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:17,752 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-10-27 23:59:17,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:17,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:17,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:17,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:17,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:17,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:17,765 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:59:17,765 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-10-27 23:59:17,765 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:17,765 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:17,765 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,765 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:17,775 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,775 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,775 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:17,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:17,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:17,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:17,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:17,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:17,776 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:17,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:17,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:17,776 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:17,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:17,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:17,785 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:17,785 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:17,785 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:17,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:17,785 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,785 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-10-27 23:59:17,785 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:17,785 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:17,785 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:17,785 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:17,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:17,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:17,785 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,785 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:17,785 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,785 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:17,785 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,886 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:17,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:17,892 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:17,894 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-10-27 23:59:17,894 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,894 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-10-27 23:59:17,894 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:17,894 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-10-27 23:59:17,894 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-10-27 23:59:17,895 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,895 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,901 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:17,901 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:17,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:17,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:17,902 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:17,902 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:17,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,903 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:17,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,903 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,903 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-10-27 23:59:17,903 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:17,903 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-10-27 23:59:17,904 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:17,904 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-10-27 23:59:17,904 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:17,904 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:17,904 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:17,904 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. started... 2023-10-27 23:59:17,904 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 738e5e9e8c019e543eb40425cbae960e 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:59:17,922 DEBUG [regionserver/jenkins-hbase16:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.46153846153846156, tune throughput to 73.08 MB/second 2023-10-27 23:59:18,328 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120abd3b73de0ec44459d26e07b709d6df2 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151120abd3b73de0ec44459d26e07b709d6df2 2023-10-27 23:59:18,737 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/.tmp/family1/67d4b0b5fc634491a86814146b850837 2023-10-27 23:59:18,748 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/.tmp/family1/67d4b0b5fc634491a86814146b850837 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/67d4b0b5fc634491a86814146b850837 2023-10-27 23:59:18,754 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/67d4b0b5fc634491a86814146b850837, entries=1, sequenceid=41, filesize=4.9 K 2023-10-27 23:59:18,755 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 738e5e9e8c019e543eb40425cbae960e in 851ms, sequenceid=41, compaction requested=true 2023-10-27 23:59:18,755 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 738e5e9e8c019e543eb40425cbae960e: 2023-10-27 23:59:18,755 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:18,755 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:18,755 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:18,756 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:18,756 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-10-27 23:59:18,756 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-10-27 23:59:18,793 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:18,793 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:18,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:18,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:18,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:18,794 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:18,794 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:18,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:18,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:18,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:18,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:18,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:18,800 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-10-27 23:59:18,800 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7dd0fd47[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:18,800 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-10-27 23:59:18,800 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:18,810 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:18,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:18,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:18,810 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-10-27 23:59:18,811 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-10-27 23:59:18,811 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:18,811 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:18,811 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:18,818 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-10-27 23:59:18,819 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:18,819 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-10-27 23:59:18,819 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:18,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:18,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:18,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:18,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:18,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:18,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:18,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:18,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:18,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:18,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:18,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:18,829 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-10-27 23:59:18,829 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-10-27 23:59:18,830 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:18,830 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:18,830 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:18,830 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:18,842 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:18,842 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:18,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:18,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:18,842 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:18,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:18,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:18,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:18,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:18,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:18,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:18,844 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:18,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:18,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:18,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:18,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:18,848 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:18,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:18,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:18,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:18,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:18,879 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:18,879 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:18,879 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:18,879 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:18,880 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-10-27 23:59:18,879 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:18,880 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:18,879 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-10-27 23:59:18,880 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:18,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:18,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:18,880 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:18,881 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:18,881 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:18,881 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:18,881 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:18,881 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:18,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:18,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:18,982 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:18,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:18,988 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36413] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-10-27 23:59:18,992 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] procedure2.ProcedureExecutor(1028): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-10-27 23:59:18,993 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-10-27 23:59:18,994 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698451758994 2023-10-27 23:59:18,994 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-27 23:59:18,996 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-10-27 23:59:18,997 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-10-27 23:59:18,997 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-10-27 23:59:18,998 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] zookeeper.ReadOnlyZKClient(139): Connect 0x0415b3ca to 127.0.0.1:57369 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-10-27 23:59:19,010 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4e79d5b3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-27 23:59:19,011 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-10-27 23:59:19,855 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103f383b8afae1b4dc99fb92d2fe491bf3a to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103f383b8afae1b4dc99fb92d2fe491bf3a 2023-10-27 23:59:19,856 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-10-27 23:59:19,867 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-10-27 23:59:19,868 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:55620, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-10-27 23:59:19,875 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-10-27 23:59:19,876 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:45246, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-10-27 23:59:19,885 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/740f85aff3f14a21a50911483adb81cc first=Optional[r3] last=Optional[r6] 2023-10-27 23:59:19,885 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/740f85aff3f14a21a50911483adb81cc}] 2023-10-27 23:59:19,890 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/740f85aff3f14a21a50911483adb81cc for inclusion in 738e5e9e8c019e543eb40425cbae960e/family1 2023-10-27 23:59:19,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-10-27 23:59:19,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(777): Region bounds: first= last= 2023-10-27 23:59:19,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for 738e5e9e8c019e543eb40425cbae960e: 2023-10-27 23:59:19,894 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/740f85aff3f14a21a50911483adb81cc to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyPolicy__40u719elkfs07kp4pv0a0q0d8b1r4elqml8b1504j0f1fjn7j7glqob3epdmt40n/family1/740f85aff3f14a21a50911483adb81cc 2023-10-27 23:59:19,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyPolicy__40u719elkfs07kp4pv0a0q0d8b1r4elqml8b1504j0f1fjn7j7glqob3epdmt40n/family1/740f85aff3f14a21a50911483adb81cc as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/aadf712c4778437b972715629a3ca2ee_SeqId_44_ 2023-10-27 23:59:19,896 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyPolicy__40u719elkfs07kp4pv0a0q0d8b1r4elqml8b1504j0f1fjn7j7glqob3epdmt40n/family1/740f85aff3f14a21a50911483adb81cc into 738e5e9e8c019e543eb40425cbae960e/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/aadf712c4778437b972715629a3ca2ee_SeqId_44_ - updating store file list. 2023-10-27 23:59:19,900 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/aadf712c4778437b972715629a3ca2ee_SeqId_44_ into 738e5e9e8c019e543eb40425cbae960e/family1 2023-10-27 23:59:19,900 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyPolicy__40u719elkfs07kp4pv0a0q0d8b1r4elqml8b1504j0f1fjn7j7glqob3epdmt40n/family1/740f85aff3f14a21a50911483adb81cc into 738e5e9e8c019e543eb40425cbae960e/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/aadf712c4778437b972715629a3ca2ee_SeqId_44_) 2023-10-27 23:59:19,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyPolicy__40u719elkfs07kp4pv0a0q0d8b1r4elqml8b1504j0f1fjn7j7glqob3epdmt40n/family1/740f85aff3f14a21a50911483adb81cc 2023-10-27 23:59:19,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-10-27 23:59:19,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-10-27 23:59:19,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(1912): 738e5e9e8c019e543eb40425cbae960e/family1 is initiating minor compaction 2023-10-27 23:59:19,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2eaa6456; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-27 23:59:19,904 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 738e5e9e8c019e543eb40425cbae960e/family1 in testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:19,904 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/5eeb20f4fd1d4cb78631e312bae38eb9, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/028cfadc4b6841359f04263a0020e571, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/38f5487b6808444bb4e6f6237e6c5023, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/8c3e3a931e434f47ad3b0d7dec5a1cbb, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/228eb7051ac446ed8a653857dec9fe4f, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/f9688645c0e748418ef4c9c559116781, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/394c603a509042bdbd382e72c6998959, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/4fc5506c47ca4d63a24bd2911adc87fe, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/67d4b0b5fc634491a86814146b850837, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/aadf712c4778437b972715629a3ca2ee_SeqId_44_] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/.tmp, totalSize=50.0 K 2023-10-27 23:59:19,905 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 5eeb20f4fd1d4cb78631e312bae38eb9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-10-27 23:59:19,905 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 028cfadc4b6841359f04263a0020e571, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-10-27 23:59:19,905 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 38f5487b6808444bb4e6f6237e6c5023, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-10-27 23:59:19,905 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 8c3e3a931e434f47ad3b0d7dec5a1cbb, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-10-27 23:59:19,905 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 228eb7051ac446ed8a653857dec9fe4f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-10-27 23:59:19,905 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting f9688645c0e748418ef4c9c559116781, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-10-27 23:59:19,905 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 394c603a509042bdbd382e72c6998959, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-10-27 23:59:19,905 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 4fc5506c47ca4d63a24bd2911adc87fe, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-10-27 23:59:19,905 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 67d4b0b5fc634491a86814146b850837, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-10-27 23:59:19,905 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting aadf712c4778437b972715629a3ca2ee_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-10-27 23:59:19,906 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-10-27 23:59:19,907 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:59:19,908 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512289212aca782d845eb974ddb32bf736caf to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512289212aca782d845eb974ddb32bf736caf 2023-10-27 23:59:19,909 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231ee57de3d1ba541fe96d23bf9bbffe758 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231ee57de3d1ba541fe96d23bf9bbffe758 2023-10-27 23:59:19,911 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101ec3206c1de154fa18d80c8916bea8e0e to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101ec3206c1de154fa18d80c8916bea8e0e 2023-10-27 23:59:19,912 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601033117933f8e4645ec9f3c2b93cd9d014b to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601033117933f8e4645ec9f3c2b93cd9d014b 2023-10-27 23:59:19,912 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-10-27 23:59:19,912 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-10-27 23:59:19,912 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-10-27 23:59:19,912 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-10-27 23:59:19,912 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-10-27 23:59:19,912 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-10-27 23:59:19,913 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-10-27 23:59:19,913 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-10-27 23:59:19,913 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-10-27 23:59:19,913 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-10-27 23:59:19,951 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] throttle.PressureAwareThroughputController(145): 738e5e9e8c019e543eb40425cbae960e#family1#compaction#49 average throughput is 1.36 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 73.08 MB/second 2023-10-27 23:59:20,771 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/.tmp/family1/9e41c48c19ee453fa05e792da54be2be as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/9e41c48c19ee453fa05e792da54be2be 2023-10-27 23:59:20,778 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 738e5e9e8c019e543eb40425cbae960e/family1 of 738e5e9e8c019e543eb40425cbae960e into 9e41c48c19ee453fa05e792da54be2be(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-10-27 23:59:20,779 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 738e5e9e8c019e543eb40425cbae960e: 2023-10-27 23:59:20,779 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1698451159904; duration=0sec 2023-10-27 23:59:20,779 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-27 23:59:20,785 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512054f464ab5eae544309c8b4dc4c92040b3 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512054f464ab5eae544309c8b4dc4c92040b3 2023-10-27 23:59:20,786 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-10-27 23:59:20,813 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9ec427a020d2447da35ee4e55b28ef9a first=Optional[r00] last=Optional[r2] 2023-10-27 23:59:20,813 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9ec427a020d2447da35ee4e55b28ef9a}] 2023-10-27 23:59:20,815 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9ec427a020d2447da35ee4e55b28ef9a for inclusion in 738e5e9e8c019e543eb40425cbae960e/family1 2023-10-27 23:59:20,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-10-27 23:59:20,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(777): Region bounds: first= last= 2023-10-27 23:59:20,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for 738e5e9e8c019e543eb40425cbae960e: 2023-10-27 23:59:20,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9ec427a020d2447da35ee4e55b28ef9a to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyPolicy__1tvf3tha3i6je91rkpd0487f4qcj5ogh1og389tpcd00p5p3m25v85m3sl4v0jkv/family1/9ec427a020d2447da35ee4e55b28ef9a 2023-10-27 23:59:20,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyPolicy__1tvf3tha3i6je91rkpd0487f4qcj5ogh1og389tpcd00p5p3m25v85m3sl4v0jkv/family1/9ec427a020d2447da35ee4e55b28ef9a as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/142824416fb94edaa2855dbcb2816624_SeqId_47_ 2023-10-27 23:59:20,823 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyPolicy__1tvf3tha3i6je91rkpd0487f4qcj5ogh1og389tpcd00p5p3m25v85m3sl4v0jkv/family1/9ec427a020d2447da35ee4e55b28ef9a into 738e5e9e8c019e543eb40425cbae960e/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/142824416fb94edaa2855dbcb2816624_SeqId_47_ - updating store file list. 2023-10-27 23:59:20,827 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/142824416fb94edaa2855dbcb2816624_SeqId_47_ into 738e5e9e8c019e543eb40425cbae960e/family1 2023-10-27 23:59:20,827 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyPolicy__1tvf3tha3i6je91rkpd0487f4qcj5ogh1og389tpcd00p5p3m25v85m3sl4v0jkv/family1/9ec427a020d2447da35ee4e55b28ef9a into 738e5e9e8c019e543eb40425cbae960e/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/142824416fb94edaa2855dbcb2816624_SeqId_47_) 2023-10-27 23:59:20,827 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyPolicy__1tvf3tha3i6je91rkpd0487f4qcj5ogh1og389tpcd00p5p3m25v85m3sl4v0jkv/family1/9ec427a020d2447da35ee4e55b28ef9a 2023-10-27 23:59:20,827 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-10-27 23:59:20,831 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 345120 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-10-27 23:59:20,831 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(1912): 738e5e9e8c019e543eb40425cbae960e/family1 is initiating minor compaction (all files) 2023-10-27 23:59:20,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7c34b5d8; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-27 23:59:20,832 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 738e5e9e8c019e543eb40425cbae960e/family1 in testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-27 23:59:20,833 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/ecf144b8ce904147b1e90baed078915a, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/9e41c48c19ee453fa05e792da54be2be, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/142824416fb94edaa2855dbcb2816624_SeqId_47_] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/.tmp, totalSize=337.0 K 2023-10-27 23:59:20,837 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting ecf144b8ce904147b1e90baed078915a, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878821089 2023-10-27 23:59:20,840 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/5eeb20f4fd1d4cb78631e312bae38eb9, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/028cfadc4b6841359f04263a0020e571, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/38f5487b6808444bb4e6f6237e6c5023, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/8c3e3a931e434f47ad3b0d7dec5a1cbb, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/228eb7051ac446ed8a653857dec9fe4f, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/f9688645c0e748418ef4c9c559116781, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/394c603a509042bdbd382e72c6998959, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/4fc5506c47ca4d63a24bd2911adc87fe, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/67d4b0b5fc634491a86814146b850837, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/aadf712c4778437b972715629a3ca2ee_SeqId_44_] to archive 2023-10-27 23:59:20,840 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 9e41c48c19ee453fa05e792da54be2be, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621221089 2023-10-27 23:59:20,840 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 142824416fb94edaa2855dbcb2816624_SeqId_47_, keycount=1002, bloomtype=ROW, size=165.4 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878821089 2023-10-27 23:59:20,841 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:59:20,841 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-10-27 23:59:20,842 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:59:20,843 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/5eeb20f4fd1d4cb78631e312bae38eb9 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/5eeb20f4fd1d4cb78631e312bae38eb9 2023-10-27 23:59:20,844 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511309a6031f1b0944ab5b0eb1e9b384a6e09 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511309a6031f1b0944ab5b0eb1e9b384a6e09 2023-10-27 23:59:20,845 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/028cfadc4b6841359f04263a0020e571 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/028cfadc4b6841359f04263a0020e571 2023-10-27 23:59:20,845 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201c538a62a67434790a22f707c0ccaca5f to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201c538a62a67434790a22f707c0ccaca5f 2023-10-27 23:59:20,846 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/38f5487b6808444bb4e6f6237e6c5023 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/38f5487b6808444bb4e6f6237e6c5023 2023-10-27 23:59:20,847 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205da9e7580e3c848648c68d283e1d940c0 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205da9e7580e3c848648c68d283e1d940c0 2023-10-27 23:59:20,847 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-10-27 23:59:20,847 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-10-27 23:59:20,847 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-10-27 23:59:20,847 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-10-27 23:59:20,848 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-27 23:59:20,849 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-10-27 23:59:20,849 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/8c3e3a931e434f47ad3b0d7dec5a1cbb to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/8c3e3a931e434f47ad3b0d7dec5a1cbb 2023-10-27 23:59:20,851 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.8590 sec 2023-10-27 23:59:20,852 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/228eb7051ac446ed8a653857dec9fe4f to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/228eb7051ac446ed8a653857dec9fe4f 2023-10-27 23:59:20,853 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/f9688645c0e748418ef4c9c559116781 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/f9688645c0e748418ef4c9c559116781 2023-10-27 23:59:20,855 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/394c603a509042bdbd382e72c6998959 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/394c603a509042bdbd382e72c6998959 2023-10-27 23:59:20,856 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/4fc5506c47ca4d63a24bd2911adc87fe to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/4fc5506c47ca4d63a24bd2911adc87fe 2023-10-27 23:59:20,857 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/67d4b0b5fc634491a86814146b850837 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/67d4b0b5fc634491a86814146b850837 2023-10-27 23:59:20,858 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/aadf712c4778437b972715629a3ca2ee_SeqId_44_ to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/aadf712c4778437b972715629a3ca2ee_SeqId_44_ 2023-10-27 23:59:20,864 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] throttle.PressureAwareThroughputController(145): 738e5e9e8c019e543eb40425cbae960e#family1#compaction#50 average throughput is 12.89 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 73.08 MB/second 2023-10-27 23:59:20,950 INFO [Listener at localhost/40295] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=248 (was 268), OpenFileDescriptor=603 (was 618), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=211 (was 250), ProcessCount=167 (was 165) - ProcessCount LEAK? -, AvailableMemoryMB=6047 (was 6054) 2023-10-27 23:59:20,960 INFO [Listener at localhost/40295] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=248, OpenFileDescriptor=603, MaxFileDescriptor=60000, SystemLoadAverage=211, ProcessCount=167, AvailableMemoryMB=6047 2023-10-27 23:59:20,963 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.HMaster$4(2112): Client=jenkins//188.40.62.62 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-10-27 23:59:20,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure2.ProcedureExecutor(1028): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:20,966 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-10-27 23:59:20,966 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(700): Client=jenkins//188.40.62.62 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-10-27 23:59:20,967 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-10-27 23:59:20,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-10-27 23:59:20,968 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a 2023-10-27 23:59:20,969 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a empty. 2023-10-27 23:59:20,969 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a 2023-10-27 23:59:20,969 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-10-27 23:59:21,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-10-27 23:59:21,270 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-10-27 23:59:21,378 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-10-27 23:59:21,380 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7675): creating {ENCODED => 5fc23723a82c912e2bb250ea7522b12a, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp 2023-10-27 23:59:21,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-10-27 23:59:21,704 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/.tmp/family1/e0e3779e875c49d28cc41864dc7524e9 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/e0e3779e875c49d28cc41864dc7524e9 2023-10-27 23:59:21,712 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 738e5e9e8c019e543eb40425cbae960e/family1 of 738e5e9e8c019e543eb40425cbae960e into e0e3779e875c49d28cc41864dc7524e9(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-10-27 23:59:21,712 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 738e5e9e8c019e543eb40425cbae960e: 2023-10-27 23:59:21,712 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e., storeName=family1, fileCount=3, fileSize=337.0 K (165.1 K, 6.6 K, 165.4 K), priority=2, time=1698451160831; duration=0sec 2023-10-27 23:59:21,712 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-27 23:59:21,744 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/ecf144b8ce904147b1e90baed078915a, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/9e41c48c19ee453fa05e792da54be2be, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/142824416fb94edaa2855dbcb2816624_SeqId_47_] to archive 2023-10-27 23:59:21,746 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:59:21,750 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/ecf144b8ce904147b1e90baed078915a to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/ecf144b8ce904147b1e90baed078915a 2023-10-27 23:59:21,752 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/9e41c48c19ee453fa05e792da54be2be to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/9e41c48c19ee453fa05e792da54be2be 2023-10-27 23:59:21,753 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/142824416fb94edaa2855dbcb2816624_SeqId_47_ to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/family1/142824416fb94edaa2855dbcb2816624_SeqId_47_ 2023-10-27 23:59:21,787 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:59:21,787 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1604): Closing 5fc23723a82c912e2bb250ea7522b12a, disabling compactions & flushes 2023-10-27 23:59:21,787 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:21,787 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:21,787 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. after waiting 0 ms 2023-10-27 23:59:21,787 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:21,787 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:21,787 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1558): Region close journal for 5fc23723a82c912e2bb250ea7522b12a: 2023-10-27 23:59:21,791 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-10-27 23:59:21,793 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1698451161793"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451161793"}]},"ts":"1698451161793"} 2023-10-27 23:59:21,796 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-10-27 23:59:21,797 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-10-27 23:59:21,797 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451161797"}]},"ts":"1698451161797"} 2023-10-27 23:59:21,799 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-10-27 23:59:21,850 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=5fc23723a82c912e2bb250ea7522b12a, ASSIGN}] 2023-10-27 23:59:21,853 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=5fc23723a82c912e2bb250ea7522b12a, ASSIGN 2023-10-27 23:59:21,855 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=5fc23723a82c912e2bb250ea7522b12a, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,43469,1698451096272; forceNewPlan=false, retain=false 2023-10-27 23:59:22,006 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=5fc23723a82c912e2bb250ea7522b12a, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:22,007 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1698451162006"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451162006"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451162006"}]},"ts":"1698451162006"} 2023-10-27 23:59:22,011 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure 5fc23723a82c912e2bb250ea7522b12a, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-27 23:59:22,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-10-27 23:59:22,166 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:22,172 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:22,172 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 5fc23723a82c912e2bb250ea7522b12a, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a.', STARTKEY => '', ENDKEY => ''} 2023-10-27 23:59:22,173 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 5fc23723a82c912e2bb250ea7522b12a 2023-10-27 23:59:22,173 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:59:22,173 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for 5fc23723a82c912e2bb250ea7522b12a 2023-10-27 23:59:22,173 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for 5fc23723a82c912e2bb250ea7522b12a 2023-10-27 23:59:22,176 INFO [StoreOpener-5fc23723a82c912e2bb250ea7522b12a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 5fc23723a82c912e2bb250ea7522b12a 2023-10-27 23:59:22,180 DEBUG [StoreOpener-5fc23723a82c912e2bb250ea7522b12a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1 2023-10-27 23:59:22,180 DEBUG [StoreOpener-5fc23723a82c912e2bb250ea7522b12a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1 2023-10-27 23:59:22,181 INFO [StoreOpener-5fc23723a82c912e2bb250ea7522b12a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5fc23723a82c912e2bb250ea7522b12a columnFamilyName family1 2023-10-27 23:59:22,184 INFO [StoreOpener-5fc23723a82c912e2bb250ea7522b12a-1] regionserver.HStore(310): Store=5fc23723a82c912e2bb250ea7522b12a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-27 23:59:22,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a 2023-10-27 23:59:22,187 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a 2023-10-27 23:59:22,192 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for 5fc23723a82c912e2bb250ea7522b12a 2023-10-27 23:59:22,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-10-27 23:59:22,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened 5fc23723a82c912e2bb250ea7522b12a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65801354, jitterRate=-0.01948341727256775}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-10-27 23:59:22,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for 5fc23723a82c912e2bb250ea7522b12a: 2023-10-27 23:59:22,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a., pid=41, masterSystemTime=1698451162166 2023-10-27 23:59:22,196 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:22,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:22,197 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=5fc23723a82c912e2bb250ea7522b12a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:22,197 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1698451162197"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451162197"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451162197"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451162197"}]},"ts":"1698451162197"} 2023-10-27 23:59:22,200 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=41, resume processing ppid=40 2023-10-27 23:59:22,200 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure 5fc23723a82c912e2bb250ea7522b12a, server=jenkins-hbase16.apache.org,43469,1698451096272 in 187 msec 2023-10-27 23:59:22,201 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=40, resume processing ppid=39 2023-10-27 23:59:22,201 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=5fc23723a82c912e2bb250ea7522b12a, ASSIGN in 350 msec 2023-10-27 23:59:22,202 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-10-27 23:59:22,202 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451162202"}]},"ts":"1698451162202"} 2023-10-27 23:59:22,203 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-10-27 23:59:22,210 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-10-27 23:59:22,212 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.2470 sec 2023-10-27 23:59:23,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-10-27 23:59:23,078 INFO [Listener at localhost/40295] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-10-27 23:59:23,111 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:23,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:23,113 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-10-27 23:59:23,113 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:23,113 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-10-27 23:59:23,113 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-10-27 23:59:23,114 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:23,114 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:23,143 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:23,143 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:23,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:23,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:23,143 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:23,144 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:23,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:23,144 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:23,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:23,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:23,145 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:23,145 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:23,145 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-10-27 23:59:23,145 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:23,145 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-10-27 23:59:23,146 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:23,146 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:23,146 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:23,146 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. started... 2023-10-27 23:59:23,147 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5fc23723a82c912e2bb250ea7522b12a 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-10-27 23:59:23,576 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130af05df5fde87490fb422b01821f53771 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130af05df5fde87490fb422b01821f53771 2023-10-27 23:59:23,924 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-10-27 23:59:23,984 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/.tmp/family1/231af7f82da9493aa2a9e7f866ee59b7 2023-10-27 23:59:23,991 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/.tmp/family1/231af7f82da9493aa2a9e7f866ee59b7 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/231af7f82da9493aa2a9e7f866ee59b7 2023-10-27 23:59:23,997 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/231af7f82da9493aa2a9e7f866ee59b7, entries=1000, sequenceid=5, filesize=191.4 K 2023-10-27 23:59:24,019 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 5fc23723a82c912e2bb250ea7522b12a in 871ms, sequenceid=5, compaction requested=false 2023-10-27 23:59:24,019 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5fc23723a82c912e2bb250ea7522b12a: 2023-10-27 23:59:24,019 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:24,019 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:24,019 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:24,019 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:24,019 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-10-27 23:59:24,019 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-10-27 23:59:24,053 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:24,053 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:24,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:24,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:24,054 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,054 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:24,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:24,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:24,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:24,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:24,056 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-10-27 23:59:24,057 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b4984b9[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:24,057 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-10-27 23:59:24,057 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,068 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,068 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-10-27 23:59:24,068 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-10-27 23:59:24,068 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:24,069 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:24,069 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:24,076 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-10-27 23:59:24,076 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:24,076 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-10-27 23:59:24,076 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:24,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:24,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:24,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:24,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:24,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:24,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:24,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:24,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:24,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:24,085 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-10-27 23:59:24,085 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-10-27 23:59:24,085 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:24,085 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:24,085 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,085 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:24,093 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,093 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:24,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:24,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,093 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:24,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:24,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:24,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:24,095 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:24,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:24,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,098 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:24,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:24,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:24,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:24,109 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:24,110 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,110 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,110 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:24,110 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-10-27 23:59:24,110 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:24,110 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:24,110 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-10-27 23:59:24,110 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,110 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:24,110 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:24,110 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:24,110 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:24,110 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:24,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:24,110 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:24,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:24,211 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:24,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:24,225 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:24,228 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-10-27 23:59:24,229 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,229 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-10-27 23:59:24,229 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:24,229 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-10-27 23:59:24,229 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-10-27 23:59:24,230 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,230 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,242 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:24,242 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:24,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:24,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:24,243 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:24,243 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:24,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,245 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:24,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,246 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,247 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-10-27 23:59:24,247 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:24,248 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-10-27 23:59:24,248 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:24,248 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-10-27 23:59:24,248 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:24,248 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:24,248 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:24,249 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. started... 2023-10-27 23:59:24,249 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5fc23723a82c912e2bb250ea7522b12a 1/1 column families, dataSize=74 B heapSize=408 B 2023-10-27 23:59:24,665 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112828c26d03d7384de5b0dec12248725c39 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112828c26d03d7384de5b0dec12248725c39 2023-10-27 23:59:25,074 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/.tmp/family1/270eb14de8304ea89ad4bfaedf357360 2023-10-27 23:59:25,085 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/.tmp/family1/270eb14de8304ea89ad4bfaedf357360 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/270eb14de8304ea89ad4bfaedf357360 2023-10-27 23:59:25,090 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/270eb14de8304ea89ad4bfaedf357360, entries=1, sequenceid=9, filesize=5.0 K 2023-10-27 23:59:25,091 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 5fc23723a82c912e2bb250ea7522b12a in 842ms, sequenceid=9, compaction requested=false 2023-10-27 23:59:25,091 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5fc23723a82c912e2bb250ea7522b12a: 2023-10-27 23:59:25,091 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:25,092 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:25,092 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:25,092 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:25,092 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-10-27 23:59:25,092 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-10-27 23:59:25,126 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:25,126 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:25,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:25,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:25,127 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,127 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:25,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:25,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:25,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:25,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:25,131 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-10-27 23:59:25,131 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2c97d2ff[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:25,131 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-10-27 23:59:25,131 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,142 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,142 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-10-27 23:59:25,142 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-10-27 23:59:25,142 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:25,142 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:25,142 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:25,151 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:25,151 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-10-27 23:59:25,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:25,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:25,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:25,151 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:25,152 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-10-27 23:59:25,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:25,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:25,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:25,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:25,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:25,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:25,158 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-10-27 23:59:25,158 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-10-27 23:59:25,158 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:25,159 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:25,159 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,159 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:25,168 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,168 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:25,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:25,168 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:25,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:25,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:25,170 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:25,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:25,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:25,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,173 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:25,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:25,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:25,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:25,204 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:25,204 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:25,204 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:25,204 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,205 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,205 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:25,205 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,205 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,205 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,204 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-10-27 23:59:25,205 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:25,204 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:25,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:25,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:25,206 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:25,206 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:25,205 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-10-27 23:59:25,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:25,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:25,306 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:25,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:25,319 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:25,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-10-27 23:59:25,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,322 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-10-27 23:59:25,322 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:25,323 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-10-27 23:59:25,323 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-10-27 23:59:25,324 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,324 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,334 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:25,335 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:25,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:25,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:25,335 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:25,335 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:25,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,336 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:25,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,336 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,336 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-10-27 23:59:25,337 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:25,337 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-10-27 23:59:25,337 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:25,337 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-10-27 23:59:25,337 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:25,337 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:25,337 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:25,337 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. started... 2023-10-27 23:59:25,337 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5fc23723a82c912e2bb250ea7522b12a 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:59:25,757 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201e7730cc8388549cb93861d6889868c55 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201e7730cc8388549cb93861d6889868c55 2023-10-27 23:59:26,167 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/.tmp/family1/c4260e5588324a88accb8dd1918f2fcf 2023-10-27 23:59:26,179 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/.tmp/family1/c4260e5588324a88accb8dd1918f2fcf as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/c4260e5588324a88accb8dd1918f2fcf 2023-10-27 23:59:26,183 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/c4260e5588324a88accb8dd1918f2fcf, entries=1, sequenceid=13, filesize=5.0 K 2023-10-27 23:59:26,184 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5fc23723a82c912e2bb250ea7522b12a in 847ms, sequenceid=13, compaction requested=true 2023-10-27 23:59:26,184 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5fc23723a82c912e2bb250ea7522b12a: 2023-10-27 23:59:26,184 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:26,184 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:26,184 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:26,184 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,184 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-10-27 23:59:26,184 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-10-27 23:59:26,218 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,218 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:26,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:26,219 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,219 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:26,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:26,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:26,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:26,222 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-10-27 23:59:26,222 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@708417f8[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:26,222 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-10-27 23:59:26,223 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,233 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,234 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-10-27 23:59:26,234 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-10-27 23:59:26,234 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:26,234 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,234 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:26,243 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-10-27 23:59:26,243 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:26,243 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-10-27 23:59:26,243 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:26,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:26,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:26,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:26,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:26,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:26,253 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-10-27 23:59:26,253 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-10-27 23:59:26,254 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:26,254 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:26,254 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,254 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:26,260 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,260 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,260 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:26,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:26,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:26,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:26,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:26,261 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:26,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:26,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,263 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:26,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,274 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,274 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,275 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,275 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:26,275 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:26,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:26,275 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:26,275 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-10-27 23:59:26,275 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-10-27 23:59:26,275 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:26,275 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:26,275 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:26,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:26,275 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,275 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:26,275 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,275 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:26,376 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:26,377 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:26,381 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:26,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-10-27 23:59:26,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,384 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-10-27 23:59:26,384 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:26,384 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-10-27 23:59:26,384 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-10-27 23:59:26,384 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,384 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,393 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,393 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:26,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:26,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:26,394 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,394 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:26,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,394 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:26,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,395 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,396 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-10-27 23:59:26,396 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,396 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-10-27 23:59:26,396 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:26,396 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-10-27 23:59:26,396 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:26,396 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:26,396 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:26,396 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. started... 2023-10-27 23:59:26,397 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5fc23723a82c912e2bb250ea7522b12a 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:59:26,414 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205f3bc990d5a3445ad97df43dfbe8c1877 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205f3bc990d5a3445ad97df43dfbe8c1877 2023-10-27 23:59:26,821 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/.tmp/family1/2f9bb21cc7e74e239c7811dcf7227c49 2023-10-27 23:59:26,828 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/.tmp/family1/2f9bb21cc7e74e239c7811dcf7227c49 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/2f9bb21cc7e74e239c7811dcf7227c49 2023-10-27 23:59:26,831 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/2f9bb21cc7e74e239c7811dcf7227c49, entries=1, sequenceid=17, filesize=5.0 K 2023-10-27 23:59:26,832 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5fc23723a82c912e2bb250ea7522b12a in 435ms, sequenceid=17, compaction requested=true 2023-10-27 23:59:26,832 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5fc23723a82c912e2bb250ea7522b12a: 2023-10-27 23:59:26,832 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:26,832 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:26,832 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:26,832 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,832 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-10-27 23:59:26,832 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-10-27 23:59:26,868 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,868 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:26,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:26,868 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:26,869 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:26,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:26,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:26,871 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-10-27 23:59:26,871 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ac4f66a[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:26,871 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-10-27 23:59:26,871 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,876 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,877 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-10-27 23:59:26,877 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-10-27 23:59:26,877 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:26,877 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,877 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:26,885 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-10-27 23:59:26,885 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:26,885 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-10-27 23:59:26,885 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:26,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:26,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:26,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:26,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:26,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:26,891 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-10-27 23:59:26,891 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-10-27 23:59:26,891 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:26,891 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:26,891 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,891 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:26,901 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,901 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:26,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:26,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,901 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:26,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:26,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:26,902 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:26,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:26,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,905 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:26,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,916 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,916 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:26,917 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,917 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,917 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:26,917 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,917 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,917 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:26,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:26,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:26,917 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:26,917 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-10-27 23:59:26,918 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:26,917 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:26,918 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:26,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:26,918 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:26,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:27,019 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:27,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:27,030 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:27,034 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-10-27 23:59:27,034 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,034 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-10-27 23:59:27,034 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:27,034 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-10-27 23:59:27,034 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-10-27 23:59:27,035 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,036 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,064 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:27,064 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:27,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:27,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:27,065 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:27,065 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:27,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,066 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:27,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,066 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,066 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-10-27 23:59:27,067 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,067 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-10-27 23:59:27,067 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:27,067 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-10-27 23:59:27,067 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:27,067 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:27,067 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:27,067 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. started... 2023-10-27 23:59:27,067 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5fc23723a82c912e2bb250ea7522b12a 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:59:27,492 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512280088c37f576947dd9f633f9c50f08d3f to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512280088c37f576947dd9f633f9c50f08d3f 2023-10-27 23:59:27,501 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/.tmp/family1/53ba3854220b48aa8065e93c654e26a2 2023-10-27 23:59:27,508 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/.tmp/family1/53ba3854220b48aa8065e93c654e26a2 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/53ba3854220b48aa8065e93c654e26a2 2023-10-27 23:59:27,512 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/53ba3854220b48aa8065e93c654e26a2, entries=1, sequenceid=21, filesize=5.0 K 2023-10-27 23:59:27,513 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5fc23723a82c912e2bb250ea7522b12a in 446ms, sequenceid=21, compaction requested=true 2023-10-27 23:59:27,513 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5fc23723a82c912e2bb250ea7522b12a: 2023-10-27 23:59:27,513 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:27,513 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:27,513 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:27,513 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:27,513 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-10-27 23:59:27,513 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-10-27 23:59:27,543 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,543 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:27,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:27,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:27,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:27,543 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,543 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:27,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:27,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:27,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:27,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:27,545 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-10-27 23:59:27,546 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5d7fc7c7[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:27,546 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-10-27 23:59:27,546 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,551 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,551 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-10-27 23:59:27,551 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-10-27 23:59:27,551 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:27,551 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:27,552 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:27,559 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:27,559 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-10-27 23:59:27,560 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:27,560 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-10-27 23:59:27,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:27,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:27,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:27,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:27,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:27,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:27,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:27,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:27,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:27,563 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-10-27 23:59:27,563 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-10-27 23:59:27,563 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:27,563 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:27,563 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,563 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:27,568 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,568 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:27,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:27,568 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:27,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:27,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:27,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:27,569 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:27,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:27,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:27,572 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:27,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:27,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:27,583 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:27,583 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,583 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,583 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:27,583 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:27,583 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:27,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:27,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:27,583 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:27,583 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-10-27 23:59:27,584 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:27,583 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,584 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,584 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:27,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:27,584 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:27,585 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:27,685 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:27,686 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:27,692 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:27,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-10-27 23:59:27,695 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,695 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-10-27 23:59:27,695 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:27,695 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-10-27 23:59:27,695 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-10-27 23:59:27,696 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,696 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,701 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:27,701 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:27,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:27,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:27,702 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:27,702 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:27,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,703 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-10-27 23:59:27,703 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:27,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,704 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,705 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-10-27 23:59:27,705 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:27,705 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-10-27 23:59:27,705 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:27,705 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-10-27 23:59:27,706 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:27,706 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:27,706 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:27,706 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. started... 2023-10-27 23:59:27,706 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5fc23723a82c912e2bb250ea7522b12a 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:59:28,132 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512311cc453bd8d5945eaab948e12284b6bee to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512311cc453bd8d5945eaab948e12284b6bee 2023-10-27 23:59:28,541 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/.tmp/family1/b3e533ae3e4946b39fea4b50f7dbe188 2023-10-27 23:59:28,548 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/.tmp/family1/b3e533ae3e4946b39fea4b50f7dbe188 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/b3e533ae3e4946b39fea4b50f7dbe188 2023-10-27 23:59:28,555 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/b3e533ae3e4946b39fea4b50f7dbe188, entries=1, sequenceid=25, filesize=5.0 K 2023-10-27 23:59:28,556 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5fc23723a82c912e2bb250ea7522b12a in 850ms, sequenceid=25, compaction requested=true 2023-10-27 23:59:28,556 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5fc23723a82c912e2bb250ea7522b12a: 2023-10-27 23:59:28,556 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:28,556 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:28,556 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:28,556 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:28,556 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-10-27 23:59:28,556 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-10-27 23:59:28,593 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:28,593 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:28,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:28,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:28,594 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,594 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:28,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:28,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:28,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:28,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:28,600 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-10-27 23:59:28,600 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4fede6bd[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:28,600 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-10-27 23:59:28,600 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,609 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,610 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-10-27 23:59:28,610 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-10-27 23:59:28,610 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:28,610 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:28,610 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:28,618 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-10-27 23:59:28,618 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:28,618 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-10-27 23:59:28,618 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:28,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:28,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:28,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:28,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:28,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:28,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:28,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:28,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:28,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:28,622 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-10-27 23:59:28,622 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-10-27 23:59:28,622 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:28,622 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:28,622 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,622 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:28,626 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,626 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:28,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:28,626 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:28,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:28,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:28,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:28,627 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:28,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:28,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:28,629 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:28,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:28,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:28,634 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:28,634 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:28,634 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:28,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:28,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:28,635 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,635 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,635 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:28,635 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,635 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-10-27 23:59:28,635 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:28,635 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:28,635 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:28,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:28,635 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-10-27 23:59:28,635 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:28,635 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,636 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:28,736 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:28,738 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:28,748 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:28,751 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-10-27 23:59:28,751 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,752 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-10-27 23:59:28,752 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:28,752 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-10-27 23:59:28,752 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-10-27 23:59:28,753 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,753 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,760 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:28,760 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:28,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:28,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:28,761 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:28,761 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:28,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,762 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:28,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,763 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,764 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-10-27 23:59:28,764 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:28,764 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-10-27 23:59:28,764 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:28,764 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-10-27 23:59:28,765 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:28,765 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:28,765 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:28,765 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. started... 2023-10-27 23:59:28,765 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5fc23723a82c912e2bb250ea7522b12a 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:59:29,195 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601013ab0839e367f4fdda59b1260c328c7e9 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601013ab0839e367f4fdda59b1260c328c7e9 2023-10-27 23:59:29,603 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/.tmp/family1/6ef1842ae83d473e9dfbd2a2d6455749 2023-10-27 23:59:29,614 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/.tmp/family1/6ef1842ae83d473e9dfbd2a2d6455749 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/6ef1842ae83d473e9dfbd2a2d6455749 2023-10-27 23:59:29,620 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/6ef1842ae83d473e9dfbd2a2d6455749, entries=1, sequenceid=29, filesize=5.0 K 2023-10-27 23:59:29,621 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5fc23723a82c912e2bb250ea7522b12a in 856ms, sequenceid=29, compaction requested=true 2023-10-27 23:59:29,621 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5fc23723a82c912e2bb250ea7522b12a: 2023-10-27 23:59:29,621 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:29,621 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:29,621 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:29,622 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:29,622 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-10-27 23:59:29,622 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-10-27 23:59:29,651 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:29,651 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:29,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:29,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:29,652 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,652 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:29,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:29,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:29,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:29,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:29,657 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-10-27 23:59:29,657 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1f06136f[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:29,657 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-10-27 23:59:29,657 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,668 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,668 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-10-27 23:59:29,669 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-10-27 23:59:29,669 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:29,669 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:29,669 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:29,691 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-10-27 23:59:29,691 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:29,691 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:29,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:29,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:29,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:29,691 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-10-27 23:59:29,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:29,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:29,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:29,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:29,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:29,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:29,702 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-10-27 23:59:29,702 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-10-27 23:59:29,702 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:29,702 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:29,702 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,702 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:29,716 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,716 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:29,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,716 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:29,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:29,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:29,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:29,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:29,718 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:29,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:29,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,721 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:29,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:29,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:29,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:29,734 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:29,734 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:29,734 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:29,735 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-10-27 23:59:29,735 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:29,735 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:29,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:29,735 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:29,735 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,735 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:29,735 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:29,735 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-10-27 23:59:29,735 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:29,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:29,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:29,736 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,736 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,736 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,836 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:29,838 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:29,848 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:29,851 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-10-27 23:59:29,851 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,852 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-10-27 23:59:29,852 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:29,852 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-10-27 23:59:29,852 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-10-27 23:59:29,853 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,853 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,859 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:29,860 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:29,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:29,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:29,860 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:29,860 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:29,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,861 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:29,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,862 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,862 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-10-27 23:59:29,862 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:29,863 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-10-27 23:59:29,863 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:29,863 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-10-27 23:59:29,863 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:29,863 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:29,863 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:29,863 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. started... 2023-10-27 23:59:29,863 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5fc23723a82c912e2bb250ea7522b12a 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:59:30,287 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103b73bf65548694d4196177c99f010fd01 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103b73bf65548694d4196177c99f010fd01 2023-10-27 23:59:30,697 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/.tmp/family1/cb8f8f592ff14ad384077f5551d5af15 2023-10-27 23:59:30,705 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/.tmp/family1/cb8f8f592ff14ad384077f5551d5af15 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/cb8f8f592ff14ad384077f5551d5af15 2023-10-27 23:59:30,712 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/cb8f8f592ff14ad384077f5551d5af15, entries=1, sequenceid=33, filesize=5.0 K 2023-10-27 23:59:30,713 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5fc23723a82c912e2bb250ea7522b12a in 850ms, sequenceid=33, compaction requested=true 2023-10-27 23:59:30,713 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5fc23723a82c912e2bb250ea7522b12a: 2023-10-27 23:59:30,714 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:30,714 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:30,714 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:30,714 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:30,714 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-10-27 23:59:30,714 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-10-27 23:59:30,743 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:30,743 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:30,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:30,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:30,744 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,744 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:30,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:30,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:30,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:30,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:30,749 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-10-27 23:59:30,749 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@32dcc143[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:30,749 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-10-27 23:59:30,749 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,760 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,760 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-10-27 23:59:30,760 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-10-27 23:59:30,760 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:30,761 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:30,761 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:30,768 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:30,768 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-10-27 23:59:30,768 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:30,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:30,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:30,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:30,769 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-10-27 23:59:30,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:30,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:30,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:30,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:30,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:30,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:30,778 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-10-27 23:59:30,778 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-10-27 23:59:30,778 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:30,778 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:30,778 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,779 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:30,784 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,784 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:30,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,784 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:30,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:30,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:30,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:30,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:30,786 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:30,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:30,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,788 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:30,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:30,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:30,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:30,801 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:30,801 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:30,801 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:30,802 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-10-27 23:59:30,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:30,802 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-10-27 23:59:30,802 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:30,802 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:30,802 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,801 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:30,802 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:30,803 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:30,802 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:30,803 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,803 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,803 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:30,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:30,903 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:30,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:30,916 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:30,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-10-27 23:59:30,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,920 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-10-27 23:59:30,920 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:30,920 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-10-27 23:59:30,921 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-10-27 23:59:30,921 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,921 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,926 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:30,926 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:30,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:30,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:30,927 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:30,927 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:30,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,928 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:30,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,929 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,929 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-10-27 23:59:30,929 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:30,929 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-10-27 23:59:30,929 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:30,929 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-10-27 23:59:30,929 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:30,929 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:30,929 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:30,930 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. started... 2023-10-27 23:59:30,930 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5fc23723a82c912e2bb250ea7522b12a 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:59:31,356 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509075a5e0b55f1a94088b4c5b4d14f978a1b to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201509075a5e0b55f1a94088b4c5b4d14f978a1b 2023-10-27 23:59:31,766 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/.tmp/family1/e90ff00c1c824d8683be1013b35d1d77 2023-10-27 23:59:31,778 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/.tmp/family1/e90ff00c1c824d8683be1013b35d1d77 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/e90ff00c1c824d8683be1013b35d1d77 2023-10-27 23:59:31,784 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/e90ff00c1c824d8683be1013b35d1d77, entries=1, sequenceid=37, filesize=5.0 K 2023-10-27 23:59:31,784 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5fc23723a82c912e2bb250ea7522b12a in 854ms, sequenceid=37, compaction requested=true 2023-10-27 23:59:31,785 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5fc23723a82c912e2bb250ea7522b12a: 2023-10-27 23:59:31,785 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:31,785 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:31,785 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:31,785 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:31,785 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-10-27 23:59:31,785 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-10-27 23:59:31,818 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:31,818 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:31,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:31,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:31,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:31,818 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:31,818 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:31,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:31,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:31,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:31,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:31,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:31,819 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-10-27 23:59:31,819 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@113e6920[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:31,819 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-10-27 23:59:31,820 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:31,826 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:31,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:31,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:31,826 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-10-27 23:59:31,826 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-10-27 23:59:31,826 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:31,826 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:31,826 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:31,834 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-10-27 23:59:31,834 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:31,834 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-10-27 23:59:31,834 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:31,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:31,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:31,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:31,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:31,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:31,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:31,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:31,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:31,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:31,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:31,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:31,839 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-10-27 23:59:31,839 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-10-27 23:59:31,839 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:31,840 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:31,840 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:31,840 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:31,849 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:31,849 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:31,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:31,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:31,849 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:31,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:31,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:31,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:31,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:31,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:31,850 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:31,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:31,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:31,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:31,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:31,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:31,851 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:31,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:31,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:31,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:31,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:31,874 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:31,874 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:31,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:31,874 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:31,874 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:31,874 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:31,874 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:31,874 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:31,874 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:31,874 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:31,874 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-10-27 23:59:31,874 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-10-27 23:59:31,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:31,874 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:31,874 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:31,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:31,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:31,875 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:31,875 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:31,975 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:31,977 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:31,987 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:31,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-10-27 23:59:31,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:31,991 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-10-27 23:59:31,991 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:31,991 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-10-27 23:59:31,991 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-10-27 23:59:31,992 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:31,992 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,001 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:32,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:32,001 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:32,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:32,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,002 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:32,002 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:32,003 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:32,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,003 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,004 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-10-27 23:59:32,004 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,004 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-10-27 23:59:32,004 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:32,004 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-10-27 23:59:32,004 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:32,004 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:32,004 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:32,005 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. started... 2023-10-27 23:59:32,005 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5fc23723a82c912e2bb250ea7522b12a 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:59:32,427 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112086dc858420dc444fbf0d74677ac98acb to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112086dc858420dc444fbf0d74677ac98acb 2023-10-27 23:59:32,835 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/.tmp/family1/c926485b5af844018b68ba849af3202a 2023-10-27 23:59:32,839 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/.tmp/family1/c926485b5af844018b68ba849af3202a as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/c926485b5af844018b68ba849af3202a 2023-10-27 23:59:32,845 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/c926485b5af844018b68ba849af3202a, entries=1, sequenceid=41, filesize=5.0 K 2023-10-27 23:59:32,846 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5fc23723a82c912e2bb250ea7522b12a in 841ms, sequenceid=41, compaction requested=true 2023-10-27 23:59:32,846 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5fc23723a82c912e2bb250ea7522b12a: 2023-10-27 23:59:32,846 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:32,846 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:32,846 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:32,846 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:32,846 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-10-27 23:59:32,847 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-10-27 23:59:32,876 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:32,876 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:32,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:32,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:32,877 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,877 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:32,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:32,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:32,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:32,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:32,878 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-10-27 23:59:32,879 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3146bdb4[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:32,879 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-10-27 23:59:32,879 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,884 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,884 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-10-27 23:59:32,884 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-10-27 23:59:32,884 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:32,885 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:32,885 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:32,893 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-10-27 23:59:32,893 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:32,893 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:32,893 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-10-27 23:59:32,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:32,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:32,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:32,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:32,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:32,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:32,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:32,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:32,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:32,897 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-10-27 23:59:32,897 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-10-27 23:59:32,897 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:32,897 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:32,897 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,897 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:32,908 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,908 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:32,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:32,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,908 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:32,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:32,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:32,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:32,909 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:32,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:32,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:32,911 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:32,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:32,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:32,918 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:32,918 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,918 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,918 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:32,918 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-10-27 23:59:32,918 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:32,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:32,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:32,918 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:32,918 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:32,918 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:32,919 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,918 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:32,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:32,919 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:32,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:32,919 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:32,919 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:33,020 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:33,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:33,024 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36413] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:33,027 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] procedure2.ProcedureExecutor(1028): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-10-27 23:59:33,028 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-10-27 23:59:33,030 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-27 23:59:33,030 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698451773029 2023-10-27 23:59:33,033 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-10-27 23:59:33,035 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-10-27 23:59:33,035 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-10-27 23:59:33,037 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] zookeeper.ReadOnlyZKClient(139): Connect 0x532035c1 to 127.0.0.1:57369 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-10-27 23:59:33,050 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7ed1d533, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-27 23:59:33,050 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-10-27 23:59:33,899 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601030abe7edfeda04d40a239271ba7adc61f to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601030abe7edfeda04d40a239271ba7adc61f 2023-10-27 23:59:33,900 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-10-27 23:59:33,915 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-10-27 23:59:33,916 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:48734, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-10-27 23:59:33,922 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-10-27 23:59:33,924 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:53020, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-10-27 23:59:33,935 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/8465eb6a8fd04327a0bb79b1480dbb54 first=Optional[r3] last=Optional[r6] 2023-10-27 23:59:33,936 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/8465eb6a8fd04327a0bb79b1480dbb54}] 2023-10-27 23:59:33,942 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/8465eb6a8fd04327a0bb79b1480dbb54 for inclusion in 5fc23723a82c912e2bb250ea7522b12a/family1 2023-10-27 23:59:33,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-10-27 23:59:33,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(777): Region bounds: first= last= 2023-10-27 23:59:33,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for 5fc23723a82c912e2bb250ea7522b12a: 2023-10-27 23:59:33,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/8465eb6a8fd04327a0bb79b1480dbb54 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__a42hr9cjh7vajk2p0hs80b1ulbcv5mp7tcpsv1kjeo1vqri2pg6ibh4ddrrquj75/family1/8465eb6a8fd04327a0bb79b1480dbb54 2023-10-27 23:59:33,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__a42hr9cjh7vajk2p0hs80b1ulbcv5mp7tcpsv1kjeo1vqri2pg6ibh4ddrrquj75/family1/8465eb6a8fd04327a0bb79b1480dbb54 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/146bedfaeed34bd7b2f788dc36175a2f_SeqId_44_ 2023-10-27 23:59:33,948 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__a42hr9cjh7vajk2p0hs80b1ulbcv5mp7tcpsv1kjeo1vqri2pg6ibh4ddrrquj75/family1/8465eb6a8fd04327a0bb79b1480dbb54 into 5fc23723a82c912e2bb250ea7522b12a/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/146bedfaeed34bd7b2f788dc36175a2f_SeqId_44_ - updating store file list. 2023-10-27 23:59:33,951 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/146bedfaeed34bd7b2f788dc36175a2f_SeqId_44_ into 5fc23723a82c912e2bb250ea7522b12a/family1 2023-10-27 23:59:33,952 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__a42hr9cjh7vajk2p0hs80b1ulbcv5mp7tcpsv1kjeo1vqri2pg6ibh4ddrrquj75/family1/8465eb6a8fd04327a0bb79b1480dbb54 into 5fc23723a82c912e2bb250ea7522b12a/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/146bedfaeed34bd7b2f788dc36175a2f_SeqId_44_) 2023-10-27 23:59:33,952 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__a42hr9cjh7vajk2p0hs80b1ulbcv5mp7tcpsv1kjeo1vqri2pg6ibh4ddrrquj75/family1/8465eb6a8fd04327a0bb79b1480dbb54 2023-10-27 23:59:33,952 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-10-27 23:59:33,956 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-10-27 23:59:33,956 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(1912): 5fc23723a82c912e2bb250ea7522b12a/family1 is initiating minor compaction 2023-10-27 23:59:33,956 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4483fe3c; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-10-27 23:59:33,956 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 5fc23723a82c912e2bb250ea7522b12a/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:33,957 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/270eb14de8304ea89ad4bfaedf357360, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/c4260e5588324a88accb8dd1918f2fcf, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/2f9bb21cc7e74e239c7811dcf7227c49, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/53ba3854220b48aa8065e93c654e26a2, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/b3e533ae3e4946b39fea4b50f7dbe188, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/6ef1842ae83d473e9dfbd2a2d6455749, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/cb8f8f592ff14ad384077f5551d5af15, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/e90ff00c1c824d8683be1013b35d1d77, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/c926485b5af844018b68ba849af3202a, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/146bedfaeed34bd7b2f788dc36175a2f_SeqId_44_] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/.tmp, totalSize=50.4 K 2023-10-27 23:59:33,957 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 270eb14de8304ea89ad4bfaedf357360, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-10-27 23:59:33,957 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting c4260e5588324a88accb8dd1918f2fcf, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-10-27 23:59:33,957 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 2f9bb21cc7e74e239c7811dcf7227c49, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-10-27 23:59:33,957 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 53ba3854220b48aa8065e93c654e26a2, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-10-27 23:59:33,957 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting b3e533ae3e4946b39fea4b50f7dbe188, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-10-27 23:59:33,957 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 6ef1842ae83d473e9dfbd2a2d6455749, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-10-27 23:59:33,957 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting cb8f8f592ff14ad384077f5551d5af15, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-10-27 23:59:33,957 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting e90ff00c1c824d8683be1013b35d1d77, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-10-27 23:59:33,958 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting c926485b5af844018b68ba849af3202a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-10-27 23:59:33,958 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 146bedfaeed34bd7b2f788dc36175a2f_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-10-27 23:59:33,959 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-10-27 23:59:33,960 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:59:33,961 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512280088c37f576947dd9f633f9c50f08d3f to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512280088c37f576947dd9f633f9c50f08d3f 2023-10-27 23:59:33,963 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512311cc453bd8d5945eaab948e12284b6bee to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512311cc453bd8d5945eaab948e12284b6bee 2023-10-27 23:59:33,965 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601013ab0839e367f4fdda59b1260c328c7e9 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601013ab0839e367f4fdda59b1260c328c7e9 2023-10-27 23:59:33,966 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103b73bf65548694d4196177c99f010fd01 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103b73bf65548694d4196177c99f010fd01 2023-10-27 23:59:33,966 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-10-27 23:59:33,966 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-10-27 23:59:33,967 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-10-27 23:59:33,967 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-10-27 23:59:33,967 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-10-27 23:59:33,967 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-10-27 23:59:33,967 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-10-27 23:59:33,967 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-10-27 23:59:33,967 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-10-27 23:59:33,967 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-10-27 23:59:34,003 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5fc23723a82c912e2bb250ea7522b12a#family1#compaction#61 average throughput is 0.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 73.08 MB/second 2023-10-27 23:59:34,839 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/.tmp/family1/7e052f20559442de800e3214a9108bd5 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/7e052f20559442de800e3214a9108bd5 2023-10-27 23:59:34,843 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205891c2b4d153243efa832646b67c2ea7a to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205891c2b4d153243efa832646b67c2ea7a 2023-10-27 23:59:34,843 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 5fc23723a82c912e2bb250ea7522b12a/family1 of 5fc23723a82c912e2bb250ea7522b12a into 7e052f20559442de800e3214a9108bd5(size=6.8 K), total size for store is 198.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-10-27 23:59:34,843 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-10-27 23:59:34,843 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 5fc23723a82c912e2bb250ea7522b12a: 2023-10-27 23:59:34,844 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1698451173956; duration=0sec 2023-10-27 23:59:34,844 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-27 23:59:34,866 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/d08336a809b04adf8aaffc2f3c26c316 first=Optional[r00] last=Optional[r2] 2023-10-27 23:59:34,867 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/d08336a809b04adf8aaffc2f3c26c316}] 2023-10-27 23:59:34,869 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/d08336a809b04adf8aaffc2f3c26c316 for inclusion in 5fc23723a82c912e2bb250ea7522b12a/family1 2023-10-27 23:59:34,872 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-10-27 23:59:34,872 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(777): Region bounds: first= last= 2023-10-27 23:59:34,872 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for 5fc23723a82c912e2bb250ea7522b12a: 2023-10-27 23:59:34,873 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/d08336a809b04adf8aaffc2f3c26c316 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__75ipl9t1mr5s86qvapmbecf1rk5ofpagqqk9jfpaktpsbdjrmi5fiqsh5j035ci7/family1/d08336a809b04adf8aaffc2f3c26c316 2023-10-27 23:59:34,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__75ipl9t1mr5s86qvapmbecf1rk5ofpagqqk9jfpaktpsbdjrmi5fiqsh5j035ci7/family1/d08336a809b04adf8aaffc2f3c26c316 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/6d00599dc5164b16b905eb077b88368b_SeqId_47_ 2023-10-27 23:59:34,877 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__75ipl9t1mr5s86qvapmbecf1rk5ofpagqqk9jfpaktpsbdjrmi5fiqsh5j035ci7/family1/d08336a809b04adf8aaffc2f3c26c316 into 5fc23723a82c912e2bb250ea7522b12a/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/6d00599dc5164b16b905eb077b88368b_SeqId_47_ - updating store file list. 2023-10-27 23:59:34,880 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/6d00599dc5164b16b905eb077b88368b_SeqId_47_ into 5fc23723a82c912e2bb250ea7522b12a/family1 2023-10-27 23:59:34,880 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__75ipl9t1mr5s86qvapmbecf1rk5ofpagqqk9jfpaktpsbdjrmi5fiqsh5j035ci7/family1/d08336a809b04adf8aaffc2f3c26c316 into 5fc23723a82c912e2bb250ea7522b12a/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/6d00599dc5164b16b905eb077b88368b_SeqId_47_) 2023-10-27 23:59:34,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__75ipl9t1mr5s86qvapmbecf1rk5ofpagqqk9jfpaktpsbdjrmi5fiqsh5j035ci7/family1/d08336a809b04adf8aaffc2f3c26c316 2023-10-27 23:59:34,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-10-27 23:59:34,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-10-27 23:59:34,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(1912): 5fc23723a82c912e2bb250ea7522b12a/family1 is initiating minor compaction (all files) 2023-10-27 23:59:34,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@5bb36ed0; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-10-27 23:59:34,882 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 5fc23723a82c912e2bb250ea7522b12a/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:34,882 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/231af7f82da9493aa2a9e7f866ee59b7, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/7e052f20559442de800e3214a9108bd5, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/6d00599dc5164b16b905eb077b88368b_SeqId_47_] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/.tmp, totalSize=390.1 K 2023-10-27 23:59:34,883 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 231af7f82da9493aa2a9e7f866ee59b7, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878821089 2023-10-27 23:59:34,883 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 7e052f20559442de800e3214a9108bd5, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621221089 2023-10-27 23:59:34,884 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 6d00599dc5164b16b905eb077b88368b_SeqId_47_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878821089 2023-10-27 23:59:34,884 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-10-27 23:59:34,885 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:59:34,886 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130af05df5fde87490fb422b01821f53771 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130af05df5fde87490fb422b01821f53771 2023-10-27 23:59:34,888 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201e7730cc8388549cb93861d6889868c55 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201e7730cc8388549cb93861d6889868c55 2023-10-27 23:59:34,889 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205f3bc990d5a3445ad97df43dfbe8c1877 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205f3bc990d5a3445ad97df43dfbe8c1877 2023-10-27 23:59:34,889 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-10-27 23:59:34,889 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-10-27 23:59:34,889 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-10-27 23:59:34,889 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-10-27 23:59:34,889 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-27 23:59:34,890 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-10-27 23:59:34,892 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.8640 sec 2023-10-27 23:59:34,904 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5fc23723a82c912e2bb250ea7522b12a#family1#compaction#62 average throughput is 15.05 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 73.08 MB/second 2023-10-27 23:59:34,939 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/270eb14de8304ea89ad4bfaedf357360, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/c4260e5588324a88accb8dd1918f2fcf, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/2f9bb21cc7e74e239c7811dcf7227c49, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/53ba3854220b48aa8065e93c654e26a2, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/b3e533ae3e4946b39fea4b50f7dbe188, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/6ef1842ae83d473e9dfbd2a2d6455749, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/cb8f8f592ff14ad384077f5551d5af15, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/e90ff00c1c824d8683be1013b35d1d77, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/c926485b5af844018b68ba849af3202a, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/146bedfaeed34bd7b2f788dc36175a2f_SeqId_44_] to archive 2023-10-27 23:59:34,940 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:59:34,942 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/270eb14de8304ea89ad4bfaedf357360 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/270eb14de8304ea89ad4bfaedf357360 2023-10-27 23:59:34,943 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/c4260e5588324a88accb8dd1918f2fcf to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/c4260e5588324a88accb8dd1918f2fcf 2023-10-27 23:59:34,945 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/2f9bb21cc7e74e239c7811dcf7227c49 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/2f9bb21cc7e74e239c7811dcf7227c49 2023-10-27 23:59:34,946 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/53ba3854220b48aa8065e93c654e26a2 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/53ba3854220b48aa8065e93c654e26a2 2023-10-27 23:59:34,947 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/b3e533ae3e4946b39fea4b50f7dbe188 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/b3e533ae3e4946b39fea4b50f7dbe188 2023-10-27 23:59:34,948 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/6ef1842ae83d473e9dfbd2a2d6455749 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/6ef1842ae83d473e9dfbd2a2d6455749 2023-10-27 23:59:34,950 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/cb8f8f592ff14ad384077f5551d5af15 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/cb8f8f592ff14ad384077f5551d5af15 2023-10-27 23:59:34,951 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/e90ff00c1c824d8683be1013b35d1d77 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/e90ff00c1c824d8683be1013b35d1d77 2023-10-27 23:59:34,952 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/c926485b5af844018b68ba849af3202a to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/c926485b5af844018b68ba849af3202a 2023-10-27 23:59:34,953 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/146bedfaeed34bd7b2f788dc36175a2f_SeqId_44_ to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/146bedfaeed34bd7b2f788dc36175a2f_SeqId_44_ 2023-10-27 23:59:34,972 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.HMaster$12(2454): Client=jenkins//188.40.62.62 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-10-27 23:59:34,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure2.ProcedureExecutor(1028): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:35,384 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-10-27 23:59:35,387 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-10-27 23:59:35,387 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-10-27 23:59:35,392 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-10-27 23:59:35,398 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=5fc23723a82c912e2bb250ea7522b12a, REOPEN/MOVE}] 2023-10-27 23:59:35,399 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=5fc23723a82c912e2bb250ea7522b12a, REOPEN/MOVE 2023-10-27 23:59:35,400 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=5fc23723a82c912e2bb250ea7522b12a, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:35,400 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1698451175400"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451175400"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451175400"}]},"ts":"1698451175400"} 2023-10-27 23:59:35,402 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 5fc23723a82c912e2bb250ea7522b12a, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-27 23:59:35,554 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:35,555 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(111): Close 5fc23723a82c912e2bb250ea7522b12a 2023-10-27 23:59:35,555 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing 5fc23723a82c912e2bb250ea7522b12a, disabling compactions & flushes 2023-10-27 23:59:35,555 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:36,144 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/.tmp/family1/50f45da87e9645eba1f25742cc32693b as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/50f45da87e9645eba1f25742cc32693b 2023-10-27 23:59:36,150 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 5fc23723a82c912e2bb250ea7522b12a/family1 of 5fc23723a82c912e2bb250ea7522b12a into 50f45da87e9645eba1f25742cc32693b(size=193.0 K), total size for store is 193.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-10-27 23:59:36,150 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 5fc23723a82c912e2bb250ea7522b12a: 2023-10-27 23:59:36,150 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:36,150 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:36,150 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a., storeName=family1, fileCount=3, fileSize=390.1 K (191.4 K, 6.8 K, 191.8 K), priority=2, time=1698451174882; duration=1sec 2023-10-27 23:59:36,150 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. after waiting 0 ms 2023-10-27 23:59:36,150 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-27 23:59:36,150 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:36,151 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/231af7f82da9493aa2a9e7f866ee59b7, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/7e052f20559442de800e3214a9108bd5, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/6d00599dc5164b16b905eb077b88368b_SeqId_47_] to archive 2023-10-27 23:59:36,152 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:59:36,154 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/231af7f82da9493aa2a9e7f866ee59b7 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/231af7f82da9493aa2a9e7f866ee59b7 2023-10-27 23:59:36,156 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/7e052f20559442de800e3214a9108bd5 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/7e052f20559442de800e3214a9108bd5 2023-10-27 23:59:36,158 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/6d00599dc5164b16b905eb077b88368b_SeqId_47_ to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/6d00599dc5164b16b905eb077b88368b_SeqId_47_ 2023-10-27 23:59:36,165 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-10-27 23:59:36,166 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:36,166 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for 5fc23723a82c912e2bb250ea7522b12a: 2023-10-27 23:59:36,166 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(3510): Not adding moved region record: 5fc23723a82c912e2bb250ea7522b12a to self. 2023-10-27 23:59:36,168 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(149): Closed 5fc23723a82c912e2bb250ea7522b12a 2023-10-27 23:59:36,169 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=5fc23723a82c912e2bb250ea7522b12a, regionState=CLOSED 2023-10-27 23:59:36,169 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1698451176169"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451176169"}]},"ts":"1698451176169"} 2023-10-27 23:59:36,173 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=46, resume processing ppid=45 2023-10-27 23:59:36,173 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 5fc23723a82c912e2bb250ea7522b12a, server=jenkins-hbase16.apache.org,43469,1698451096272 in 769 msec 2023-10-27 23:59:36,173 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=5fc23723a82c912e2bb250ea7522b12a, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase16.apache.org,43469,1698451096272; forceNewPlan=false, retain=true 2023-10-27 23:59:36,324 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=5fc23723a82c912e2bb250ea7522b12a, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:36,324 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1698451176324"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451176324"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451176324"}]},"ts":"1698451176324"} 2023-10-27 23:59:36,327 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure 5fc23723a82c912e2bb250ea7522b12a, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-27 23:59:36,481 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:36,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:36,487 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 5fc23723a82c912e2bb250ea7522b12a, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a.', STARTKEY => '', ENDKEY => ''} 2023-10-27 23:59:36,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 5fc23723a82c912e2bb250ea7522b12a 2023-10-27 23:59:36,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:59:36,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for 5fc23723a82c912e2bb250ea7522b12a 2023-10-27 23:59:36,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for 5fc23723a82c912e2bb250ea7522b12a 2023-10-27 23:59:36,489 INFO [StoreOpener-5fc23723a82c912e2bb250ea7522b12a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 5fc23723a82c912e2bb250ea7522b12a 2023-10-27 23:59:36,490 DEBUG [StoreOpener-5fc23723a82c912e2bb250ea7522b12a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1 2023-10-27 23:59:36,490 DEBUG [StoreOpener-5fc23723a82c912e2bb250ea7522b12a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1 2023-10-27 23:59:36,491 INFO [StoreOpener-5fc23723a82c912e2bb250ea7522b12a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5fc23723a82c912e2bb250ea7522b12a columnFamilyName family1 2023-10-27 23:59:36,495 DEBUG [StoreOpener-5fc23723a82c912e2bb250ea7522b12a-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/50f45da87e9645eba1f25742cc32693b 2023-10-27 23:59:36,495 INFO [StoreOpener-5fc23723a82c912e2bb250ea7522b12a-1] regionserver.HStore(310): Store=5fc23723a82c912e2bb250ea7522b12a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-27 23:59:36,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a 2023-10-27 23:59:36,497 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a 2023-10-27 23:59:36,501 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for 5fc23723a82c912e2bb250ea7522b12a 2023-10-27 23:59:36,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened 5fc23723a82c912e2bb250ea7522b12a; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64376977, jitterRate=-0.04070828855037689}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-10-27 23:59:36,502 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for 5fc23723a82c912e2bb250ea7522b12a: 2023-10-27 23:59:36,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a., pid=47, masterSystemTime=1698451176481 2023-10-27 23:59:36,504 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:36,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:36,505 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=5fc23723a82c912e2bb250ea7522b12a, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:36,505 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1698451176504"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451176504"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451176504"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451176504"}]},"ts":"1698451176504"} 2023-10-27 23:59:36,508 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=47, resume processing ppid=45 2023-10-27 23:59:36,508 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure 5fc23723a82c912e2bb250ea7522b12a, server=jenkins-hbase16.apache.org,43469,1698451096272 in 179 msec 2023-10-27 23:59:36,510 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=45, resume processing ppid=44 2023-10-27 23:59:36,510 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=5fc23723a82c912e2bb250ea7522b12a, REOPEN/MOVE in 1.1100 sec 2023-10-27 23:59:36,512 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=44, resume processing ppid=43 2023-10-27 23:59:36,512 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.1190 sec 2023-10-27 23:59:36,514 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.5410 sec 2023-10-27 23:59:36,515 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=43 2023-10-27 23:59:36,516 INFO [Listener at localhost/40295] compactions.TestMobCompactor(278): alter status finished 2023-10-27 23:59:36,517 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36413] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:36,518 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] procedure2.ProcedureExecutor(1028): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-10-27 23:59:36,519 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-10-27 23:59:36,519 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698451776519 2023-10-27 23:59:36,519 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-27 23:59:36,520 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-10-27 23:59:36,521 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-10-27 23:59:36,521 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-10-27 23:59:36,523 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] zookeeper.ReadOnlyZKClient(139): Connect 0x162e014f to 127.0.0.1:57369 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-10-27 23:59:36,577 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@66a626f4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-27 23:59:36,577 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-10-27 23:59:36,578 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-10-27 23:59:36,578 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-10-27 23:59:36,578 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-10-27 23:59:36,578 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-10-27 23:59:37,430 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128fe7a2dffe95a468781693b3166c5829f to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128fe7a2dffe95a468781693b3166c5829f 2023-10-27 23:59:37,431 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-10-27 23:59:37,443 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-10-27 23:59:37,444 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:48748, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-10-27 23:59:37,450 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-10-27 23:59:37,452 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:53028, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-10-27 23:59:37,460 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a6cf28ef951c48cdb1c7dcccd3eb4330 first=Optional[r05] last=Optional[r8] 2023-10-27 23:59:37,461 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=51 for row with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a6cf28ef951c48cdb1c7dcccd3eb4330}] 2023-10-27 23:59:37,466 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a6cf28ef951c48cdb1c7dcccd3eb4330 for inclusion in 5fc23723a82c912e2bb250ea7522b12a/family1 2023-10-27 23:59:37,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-10-27 23:59:37,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(777): Region bounds: first= last= 2023-10-27 23:59:37,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for 5fc23723a82c912e2bb250ea7522b12a: 2023-10-27 23:59:37,470 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a6cf28ef951c48cdb1c7dcccd3eb4330 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__1dp0cpigrjmesd2gpq3mqisd24g42f5ddlavb6b8v7pfrhd8h9uquuuj66sa8a1b/family1/a6cf28ef951c48cdb1c7dcccd3eb4330 2023-10-27 23:59:37,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__1dp0cpigrjmesd2gpq3mqisd24g42f5ddlavb6b8v7pfrhd8h9uquuuj66sa8a1b/family1/a6cf28ef951c48cdb1c7dcccd3eb4330 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/9f9c84c487494087bdb187c807e543e8_SeqId_53_ 2023-10-27 23:59:37,473 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__1dp0cpigrjmesd2gpq3mqisd24g42f5ddlavb6b8v7pfrhd8h9uquuuj66sa8a1b/family1/a6cf28ef951c48cdb1c7dcccd3eb4330 into 5fc23723a82c912e2bb250ea7522b12a/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/9f9c84c487494087bdb187c807e543e8_SeqId_53_ - updating store file list. 2023-10-27 23:59:37,477 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/9f9c84c487494087bdb187c807e543e8_SeqId_53_ into 5fc23723a82c912e2bb250ea7522b12a/family1 2023-10-27 23:59:37,477 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__1dp0cpigrjmesd2gpq3mqisd24g42f5ddlavb6b8v7pfrhd8h9uquuuj66sa8a1b/family1/a6cf28ef951c48cdb1c7dcccd3eb4330 into 5fc23723a82c912e2bb250ea7522b12a/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/9f9c84c487494087bdb187c807e543e8_SeqId_53_) 2023-10-27 23:59:37,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__1dp0cpigrjmesd2gpq3mqisd24g42f5ddlavb6b8v7pfrhd8h9uquuuj66sa8a1b/family1/a6cf28ef951c48cdb1c7dcccd3eb4330 2023-10-27 23:59:37,480 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-10-27 23:59:37,481 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:59:37,482 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112086dc858420dc444fbf0d74677ac98acb to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112086dc858420dc444fbf0d74677ac98acb 2023-10-27 23:59:37,483 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112828c26d03d7384de5b0dec12248725c39 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112828c26d03d7384de5b0dec12248725c39 2023-10-27 23:59:37,483 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-10-27 23:59:37,483 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-10-27 23:59:37,484 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-10-27 23:59:37,484 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-10-27 23:59:37,484 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-10-27 23:59:37,484 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-10-27 23:59:37,484 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-27 23:59:37,485 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-10-27 23:59:37,486 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 968 msec 2023-10-27 23:59:37,540 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.HMaster$12(2454): Client=jenkins//188.40.62.62 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-10-27 23:59:37,541 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure2.ProcedureExecutor(1028): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:37,953 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-10-27 23:59:37,956 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-10-27 23:59:37,956 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-10-27 23:59:37,959 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-10-27 23:59:37,963 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=5fc23723a82c912e2bb250ea7522b12a, REOPEN/MOVE}] 2023-10-27 23:59:37,964 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=5fc23723a82c912e2bb250ea7522b12a, REOPEN/MOVE 2023-10-27 23:59:37,965 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=5fc23723a82c912e2bb250ea7522b12a, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:37,965 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1698451177965"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451177965"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451177965"}]},"ts":"1698451177965"} 2023-10-27 23:59:37,967 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure 5fc23723a82c912e2bb250ea7522b12a, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-27 23:59:38,119 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:38,120 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(111): Close 5fc23723a82c912e2bb250ea7522b12a 2023-10-27 23:59:38,120 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing 5fc23723a82c912e2bb250ea7522b12a, disabling compactions & flushes 2023-10-27 23:59:38,120 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:38,120 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:38,120 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. after waiting 0 ms 2023-10-27 23:59:38,120 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:38,135 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-10-27 23:59:38,136 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:38,136 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for 5fc23723a82c912e2bb250ea7522b12a: 2023-10-27 23:59:38,136 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(3510): Not adding moved region record: 5fc23723a82c912e2bb250ea7522b12a to self. 2023-10-27 23:59:38,138 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(149): Closed 5fc23723a82c912e2bb250ea7522b12a 2023-10-27 23:59:38,138 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=5fc23723a82c912e2bb250ea7522b12a, regionState=CLOSED 2023-10-27 23:59:38,138 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1698451178138"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451178138"}]},"ts":"1698451178138"} 2023-10-27 23:59:38,142 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=52, resume processing ppid=51 2023-10-27 23:59:38,142 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure 5fc23723a82c912e2bb250ea7522b12a, server=jenkins-hbase16.apache.org,43469,1698451096272 in 173 msec 2023-10-27 23:59:38,142 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=5fc23723a82c912e2bb250ea7522b12a, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase16.apache.org,43469,1698451096272; forceNewPlan=false, retain=true 2023-10-27 23:59:38,293 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=5fc23723a82c912e2bb250ea7522b12a, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:38,294 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1698451178293"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451178293"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451178293"}]},"ts":"1698451178293"} 2023-10-27 23:59:38,297 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure 5fc23723a82c912e2bb250ea7522b12a, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-27 23:59:38,451 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:38,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:38,454 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 5fc23723a82c912e2bb250ea7522b12a, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a.', STARTKEY => '', ENDKEY => ''} 2023-10-27 23:59:38,454 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 5fc23723a82c912e2bb250ea7522b12a 2023-10-27 23:59:38,454 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:59:38,454 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for 5fc23723a82c912e2bb250ea7522b12a 2023-10-27 23:59:38,454 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for 5fc23723a82c912e2bb250ea7522b12a 2023-10-27 23:59:38,455 INFO [StoreOpener-5fc23723a82c912e2bb250ea7522b12a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 5fc23723a82c912e2bb250ea7522b12a 2023-10-27 23:59:38,457 DEBUG [StoreOpener-5fc23723a82c912e2bb250ea7522b12a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1 2023-10-27 23:59:38,457 DEBUG [StoreOpener-5fc23723a82c912e2bb250ea7522b12a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1 2023-10-27 23:59:38,457 INFO [StoreOpener-5fc23723a82c912e2bb250ea7522b12a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5fc23723a82c912e2bb250ea7522b12a columnFamilyName family1 2023-10-27 23:59:38,463 DEBUG [StoreOpener-5fc23723a82c912e2bb250ea7522b12a-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/50f45da87e9645eba1f25742cc32693b 2023-10-27 23:59:38,467 DEBUG [StoreOpener-5fc23723a82c912e2bb250ea7522b12a-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/family1/9f9c84c487494087bdb187c807e543e8_SeqId_53_ 2023-10-27 23:59:38,467 INFO [StoreOpener-5fc23723a82c912e2bb250ea7522b12a-1] regionserver.HStore(310): Store=5fc23723a82c912e2bb250ea7522b12a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-27 23:59:38,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a 2023-10-27 23:59:38,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a 2023-10-27 23:59:38,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for 5fc23723a82c912e2bb250ea7522b12a 2023-10-27 23:59:38,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened 5fc23723a82c912e2bb250ea7522b12a; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59429372, jitterRate=-0.11443334817886353}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-10-27 23:59:38,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for 5fc23723a82c912e2bb250ea7522b12a: 2023-10-27 23:59:38,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a., pid=53, masterSystemTime=1698451178451 2023-10-27 23:59:38,474 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:38,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-27 23:59:38,475 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=5fc23723a82c912e2bb250ea7522b12a, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:38,475 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1698451178474"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451178474"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451178474"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451178474"}]},"ts":"1698451178474"} 2023-10-27 23:59:38,477 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=53, resume processing ppid=51 2023-10-27 23:59:38,477 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure 5fc23723a82c912e2bb250ea7522b12a, server=jenkins-hbase16.apache.org,43469,1698451096272 in 179 msec 2023-10-27 23:59:38,478 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=51, resume processing ppid=50 2023-10-27 23:59:38,478 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=5fc23723a82c912e2bb250ea7522b12a, REOPEN/MOVE in 514 msec 2023-10-27 23:59:38,480 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=50, resume processing ppid=49 2023-10-27 23:59:38,480 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 520 msec 2023-10-27 23:59:38,482 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 941 msec 2023-10-27 23:59:38,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-10-27 23:59:38,483 INFO [Listener at localhost/40295] compactions.TestMobCompactor(278): alter status finished 2023-10-27 23:59:38,484 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36413] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-10-27 23:59:38,485 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] procedure2.ProcedureExecutor(1028): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-10-27 23:59:38,486 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-10-27 23:59:38,487 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698451778487 2023-10-27 23:59:38,487 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-27 23:59:38,487 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-10-27 23:59:38,488 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-10-27 23:59:38,488 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-10-27 23:59:38,489 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] zookeeper.ReadOnlyZKClient(139): Connect 0x7554ee49 to 127.0.0.1:57369 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-10-27 23:59:38,527 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2292e791, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-27 23:59:38,528 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-10-27 23:59:38,528 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-10-27 23:59:38,528 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-10-27 23:59:38,528 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-10-27 23:59:38,528 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-10-27 23:59:38,528 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-10-27 23:59:38,528 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-10-27 23:59:38,528 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-10-27 23:59:38,528 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-10-27 23:59:38,528 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-27 23:59:38,531 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-10-27 23:59:38,534 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 47 msec 2023-10-27 23:59:38,594 INFO [Listener at localhost/40295] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=249 (was 248) - Thread LEAK? -, OpenFileDescriptor=626 (was 603) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=165 (was 211), ProcessCount=166 (was 167), AvailableMemoryMB=6037 (was 6047) 2023-10-27 23:59:38,606 INFO [Listener at localhost/40295] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=249, OpenFileDescriptor=626, MaxFileDescriptor=60000, SystemLoadAverage=165, ProcessCount=166, AvailableMemoryMB=6036 2023-10-27 23:59:38,608 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.HMaster$4(2112): Client=jenkins//188.40.62.62 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-10-27 23:59:38,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure2.ProcedureExecutor(1028): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:38,610 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-10-27 23:59:38,610 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(700): Client=jenkins//188.40.62.62 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-10-27 23:59:38,610 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-10-27 23:59:38,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-10-27 23:59:38,612 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8 2023-10-27 23:59:38,613 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8 empty. 2023-10-27 23:59:38,613 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8 2023-10-27 23:59:38,613 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-10-27 23:59:38,712 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-10-27 23:59:38,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-10-27 23:59:39,026 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-10-27 23:59:39,029 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7675): creating {ENCODED => bec4bce8b8bdfe22904d678de5936bd8, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp 2023-10-27 23:59:39,217 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-10-27 23:59:39,442 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:59:39,443 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1604): Closing bec4bce8b8bdfe22904d678de5936bd8, disabling compactions & flushes 2023-10-27 23:59:39,443 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:39,443 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:39,443 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. after waiting 0 ms 2023-10-27 23:59:39,443 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:39,443 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:39,443 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1558): Region close journal for bec4bce8b8bdfe22904d678de5936bd8: 2023-10-27 23:59:39,450 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-10-27 23:59:39,452 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1698451179451"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451179451"}]},"ts":"1698451179451"} 2023-10-27 23:59:39,453 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-10-27 23:59:39,453 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-10-27 23:59:39,454 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451179453"}]},"ts":"1698451179453"} 2023-10-27 23:59:39,455 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-10-27 23:59:39,493 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=bec4bce8b8bdfe22904d678de5936bd8, ASSIGN}] 2023-10-27 23:59:39,495 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=bec4bce8b8bdfe22904d678de5936bd8, ASSIGN 2023-10-27 23:59:39,496 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=bec4bce8b8bdfe22904d678de5936bd8, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,43469,1698451096272; forceNewPlan=false, retain=false 2023-10-27 23:59:39,647 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=bec4bce8b8bdfe22904d678de5936bd8, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:39,647 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1698451179647"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451179647"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451179647"}]},"ts":"1698451179647"} 2023-10-27 23:59:39,649 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure bec4bce8b8bdfe22904d678de5936bd8, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-27 23:59:39,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-10-27 23:59:39,800 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:39,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:39,808 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => bec4bce8b8bdfe22904d678de5936bd8, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8.', STARTKEY => '', ENDKEY => ''} 2023-10-27 23:59:39,809 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily bec4bce8b8bdfe22904d678de5936bd8 2023-10-27 23:59:39,809 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:59:39,809 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for bec4bce8b8bdfe22904d678de5936bd8 2023-10-27 23:59:39,809 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for bec4bce8b8bdfe22904d678de5936bd8 2023-10-27 23:59:39,811 INFO [StoreOpener-bec4bce8b8bdfe22904d678de5936bd8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region bec4bce8b8bdfe22904d678de5936bd8 2023-10-27 23:59:39,813 DEBUG [StoreOpener-bec4bce8b8bdfe22904d678de5936bd8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1 2023-10-27 23:59:39,813 DEBUG [StoreOpener-bec4bce8b8bdfe22904d678de5936bd8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1 2023-10-27 23:59:39,814 INFO [StoreOpener-bec4bce8b8bdfe22904d678de5936bd8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bec4bce8b8bdfe22904d678de5936bd8 columnFamilyName family1 2023-10-27 23:59:39,815 INFO [StoreOpener-bec4bce8b8bdfe22904d678de5936bd8-1] regionserver.HStore(310): Store=bec4bce8b8bdfe22904d678de5936bd8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-27 23:59:39,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8 2023-10-27 23:59:39,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8 2023-10-27 23:59:39,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for bec4bce8b8bdfe22904d678de5936bd8 2023-10-27 23:59:39,821 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-10-27 23:59:39,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened bec4bce8b8bdfe22904d678de5936bd8; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68522481, jitterRate=0.021064534783363342}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-10-27 23:59:39,821 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for bec4bce8b8bdfe22904d678de5936bd8: 2023-10-27 23:59:39,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8., pid=57, masterSystemTime=1698451179800 2023-10-27 23:59:39,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:39,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:39,824 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=bec4bce8b8bdfe22904d678de5936bd8, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:39,824 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1698451179824"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451179824"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451179824"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451179824"}]},"ts":"1698451179824"} 2023-10-27 23:59:39,827 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=57, resume processing ppid=56 2023-10-27 23:59:39,827 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure bec4bce8b8bdfe22904d678de5936bd8, server=jenkins-hbase16.apache.org,43469,1698451096272 in 176 msec 2023-10-27 23:59:39,828 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=56, resume processing ppid=55 2023-10-27 23:59:39,828 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=bec4bce8b8bdfe22904d678de5936bd8, ASSIGN in 334 msec 2023-10-27 23:59:39,829 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-10-27 23:59:39,829 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451179829"}]},"ts":"1698451179829"} 2023-10-27 23:59:39,830 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-10-27 23:59:39,860 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-10-27 23:59:39,862 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.2520 sec 2023-10-27 23:59:40,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-10-27 23:59:40,723 INFO [Listener at localhost/40295] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-10-27 23:59:40,745 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:40,747 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:40,747 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-10-27 23:59:40,747 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:40,747 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-10-27 23:59:40,747 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-10-27 23:59:40,748 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:40,748 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:40,776 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:40,776 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:40,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:40,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:40,776 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:40,776 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:40,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:40,777 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:40,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:40,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:40,777 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:40,777 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:40,777 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-10-27 23:59:40,777 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:40,777 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-10-27 23:59:40,778 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:40,778 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:40,778 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:40,778 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. started... 2023-10-27 23:59:40,778 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bec4bce8b8bdfe22904d678de5936bd8 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-10-27 23:59:41,198 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130be3fc91ca8b145079380a7623cce125f to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130be3fc91ca8b145079380a7623cce125f 2023-10-27 23:59:41,602 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/.tmp/family1/464183528ba54a72980a4158ab1c2468 2023-10-27 23:59:41,615 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/.tmp/family1/464183528ba54a72980a4158ab1c2468 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/464183528ba54a72980a4158ab1c2468 2023-10-27 23:59:41,620 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/464183528ba54a72980a4158ab1c2468, entries=1000, sequenceid=5, filesize=190.5 K 2023-10-27 23:59:41,621 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for bec4bce8b8bdfe22904d678de5936bd8 in 843ms, sequenceid=5, compaction requested=false 2023-10-27 23:59:41,621 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-10-27 23:59:41,622 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bec4bce8b8bdfe22904d678de5936bd8: 2023-10-27 23:59:41,622 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:41,622 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:41,622 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:41,622 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:41,622 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-10-27 23:59:41,622 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-10-27 23:59:41,659 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:41,659 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:41,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:41,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:41,659 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,660 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:41,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:41,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:41,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:41,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:41,662 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-10-27 23:59:41,662 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1ef45d98[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:41,662 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-10-27 23:59:41,663 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,667 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,668 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-10-27 23:59:41,668 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-10-27 23:59:41,668 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:41,668 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:41,668 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:41,676 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-10-27 23:59:41,676 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:41,676 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:41,676 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-10-27 23:59:41,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:41,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:41,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:41,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:41,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:41,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:41,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:41,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:41,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:41,683 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-10-27 23:59:41,683 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-10-27 23:59:41,684 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:41,684 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:41,684 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,684 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:41,692 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,692 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,693 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:41,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:41,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:41,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:41,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:41,694 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:41,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:41,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:41,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,698 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:41,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:41,699 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-10-27 23:59:41,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:41,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:41,726 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:41,726 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:41,727 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:41,727 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-10-27 23:59:41,727 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:41,727 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-10-27 23:59:41,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:41,727 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:41,727 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,727 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:41,727 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:41,727 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,727 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,727 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:41,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:41,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:41,727 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,727 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:41,827 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:41,829 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:41,833 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:41,834 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-10-27 23:59:41,834 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,835 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-10-27 23:59:41,835 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:41,835 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-10-27 23:59:41,835 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-10-27 23:59:41,835 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,835 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,842 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:41,842 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:41,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:41,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:41,843 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:41,843 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:41,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,843 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:41,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,843 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,844 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-10-27 23:59:41,844 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:41,844 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-10-27 23:59:41,844 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:41,844 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-10-27 23:59:41,844 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:41,844 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:41,844 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:41,844 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. started... 2023-10-27 23:59:41,844 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bec4bce8b8bdfe22904d678de5936bd8 1/1 column families, dataSize=74 B heapSize=408 B 2023-10-27 23:59:42,264 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128bb9aa9dd76ac44338809008402709ffe to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128bb9aa9dd76ac44338809008402709ffe 2023-10-27 23:59:42,674 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/.tmp/family1/a17f44acaec44421b1db8fb7e8bcba36 2023-10-27 23:59:42,685 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/.tmp/family1/a17f44acaec44421b1db8fb7e8bcba36 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/a17f44acaec44421b1db8fb7e8bcba36 2023-10-27 23:59:42,690 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/a17f44acaec44421b1db8fb7e8bcba36, entries=1, sequenceid=9, filesize=5.0 K 2023-10-27 23:59:42,691 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for bec4bce8b8bdfe22904d678de5936bd8 in 847ms, sequenceid=9, compaction requested=false 2023-10-27 23:59:42,691 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bec4bce8b8bdfe22904d678de5936bd8: 2023-10-27 23:59:42,691 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:42,691 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:42,691 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:42,691 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:42,691 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-10-27 23:59:42,691 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-10-27 23:59:42,701 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:42,701 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:42,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:42,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:42,701 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,701 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:42,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:42,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:42,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:42,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:42,704 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-10-27 23:59:42,704 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4a7641dc[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:42,704 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-10-27 23:59:42,704 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,709 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,709 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-10-27 23:59:42,710 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-10-27 23:59:42,710 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:42,710 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:42,710 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:42,717 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-10-27 23:59:42,718 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:42,717 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:42,718 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-10-27 23:59:42,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:42,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:42,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:42,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:42,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:42,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:42,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:42,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:42,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:42,726 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-10-27 23:59:42,726 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-10-27 23:59:42,726 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:42,726 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:42,726 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,726 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:42,734 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,734 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,734 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:42,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:42,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:42,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:42,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:42,736 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:42,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:42,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:42,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,739 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:42,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:42,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:42,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:42,751 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:42,751 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:42,751 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:42,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:42,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:42,751 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-10-27 23:59:42,751 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-10-27 23:59:42,751 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:42,751 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:42,751 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,751 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:42,752 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:42,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:42,752 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:42,752 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:42,752 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,752 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,752 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,852 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:42,853 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:42,859 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:42,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-10-27 23:59:42,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,861 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-10-27 23:59:42,862 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:42,862 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-10-27 23:59:42,862 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-10-27 23:59:42,863 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,863 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,867 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:42,867 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:42,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:42,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:42,868 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:42,869 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:42,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,869 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:42,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,870 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,871 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-10-27 23:59:42,871 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:42,871 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-10-27 23:59:42,871 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:42,871 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-10-27 23:59:42,872 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:42,872 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:42,872 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:42,872 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. started... 2023-10-27 23:59:42,872 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bec4bce8b8bdfe22904d678de5936bd8 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:59:43,302 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512018db9338f82964fbdbd524281f93f200a to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512018db9338f82964fbdbd524281f93f200a 2023-10-27 23:59:43,710 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/.tmp/family1/cf16caabeddf43df9b074645f7a54fdb 2023-10-27 23:59:43,722 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/.tmp/family1/cf16caabeddf43df9b074645f7a54fdb as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/cf16caabeddf43df9b074645f7a54fdb 2023-10-27 23:59:43,728 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/cf16caabeddf43df9b074645f7a54fdb, entries=1, sequenceid=13, filesize=5.0 K 2023-10-27 23:59:43,729 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bec4bce8b8bdfe22904d678de5936bd8 in 857ms, sequenceid=13, compaction requested=true 2023-10-27 23:59:43,729 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bec4bce8b8bdfe22904d678de5936bd8: 2023-10-27 23:59:43,729 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:43,729 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:43,729 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:43,729 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:43,729 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-10-27 23:59:43,729 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-10-27 23:59:43,759 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:43,759 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:43,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:43,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:43,760 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,760 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:43,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:43,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:43,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:43,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:43,761 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-10-27 23:59:43,761 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-10-27 23:59:43,762 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,761 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@28e346db[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:43,767 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,767 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-10-27 23:59:43,767 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-10-27 23:59:43,767 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:43,767 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:43,768 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:43,775 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-10-27 23:59:43,775 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:43,775 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:43,776 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-10-27 23:59:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:43,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:43,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:43,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:43,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:43,779 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-10-27 23:59:43,779 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-10-27 23:59:43,779 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:43,779 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:43,779 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,779 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:43,784 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,784 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:43,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,784 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:43,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:43,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:43,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:43,785 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:43,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:43,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:43,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,786 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:43,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:43,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:43,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:43,792 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:43,792 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:43,792 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,792 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:43,792 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-10-27 23:59:43,793 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:43,792 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:43,792 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:43,793 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:43,793 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:43,793 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-10-27 23:59:43,793 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,793 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,793 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:43,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:43,793 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:43,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:43,894 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:43,895 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:43,900 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:43,902 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-10-27 23:59:43,903 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,903 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-10-27 23:59:43,903 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:43,903 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-10-27 23:59:43,903 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-10-27 23:59:43,903 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,904 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,909 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:43,909 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:43,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:43,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:43,909 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:43,909 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:43,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,910 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:43,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,910 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,911 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-10-27 23:59:43,911 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:43,911 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-10-27 23:59:43,911 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:43,911 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-10-27 23:59:43,911 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:43,911 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:43,911 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:43,911 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. started... 2023-10-27 23:59:43,911 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bec4bce8b8bdfe22904d678de5936bd8 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:59:44,332 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512058f06e4a26a454bc0b4f55be8aa56e0aa to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512058f06e4a26a454bc0b4f55be8aa56e0aa 2023-10-27 23:59:44,741 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/.tmp/family1/1fe844c9466a4aa4b7abab315c09a72a 2023-10-27 23:59:44,752 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/.tmp/family1/1fe844c9466a4aa4b7abab315c09a72a as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/1fe844c9466a4aa4b7abab315c09a72a 2023-10-27 23:59:44,757 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/1fe844c9466a4aa4b7abab315c09a72a, entries=1, sequenceid=17, filesize=5.0 K 2023-10-27 23:59:44,758 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bec4bce8b8bdfe22904d678de5936bd8 in 847ms, sequenceid=17, compaction requested=true 2023-10-27 23:59:44,758 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bec4bce8b8bdfe22904d678de5936bd8: 2023-10-27 23:59:44,758 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:44,758 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:44,758 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:44,758 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:44,758 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-10-27 23:59:44,758 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-10-27 23:59:44,811 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:44,811 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:44,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:44,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:44,812 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,812 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:44,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:44,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:44,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:44,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:44,817 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-10-27 23:59:44,817 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2cccf56e[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:44,817 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-10-27 23:59:44,818 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,826 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,826 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-10-27 23:59:44,826 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-10-27 23:59:44,826 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:44,827 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:44,827 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:44,834 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-10-27 23:59:44,834 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:44,834 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:44,834 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-10-27 23:59:44,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:44,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:44,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:44,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:44,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:44,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:44,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:44,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:44,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:44,845 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-10-27 23:59:44,845 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-10-27 23:59:44,845 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:44,845 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:44,845 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,845 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:44,850 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,850 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,851 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:44,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:44,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:44,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:44,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:44,852 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:44,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:44,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:44,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,853 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:44,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:44,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:44,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:44,866 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:44,866 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,866 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,866 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:44,866 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,866 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,866 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,866 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:44,866 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-10-27 23:59:44,866 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:44,866 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:44,867 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-10-27 23:59:44,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:44,867 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:44,866 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:44,867 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:44,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:44,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:44,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:44,968 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:44,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:44,972 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:44,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-10-27 23:59:44,974 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,974 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-10-27 23:59:44,974 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:44,974 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-10-27 23:59:44,974 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-10-27 23:59:44,974 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,974 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,984 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:44,984 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:44,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:44,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:44,984 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:44,984 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:44,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,985 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:44,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,985 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,986 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-10-27 23:59:44,986 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:44,986 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-10-27 23:59:44,986 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:44,986 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-10-27 23:59:44,986 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:44,986 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:44,986 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:44,986 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. started... 2023-10-27 23:59:44,986 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bec4bce8b8bdfe22904d678de5936bd8 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:59:45,407 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122849871a825421444a921e03f1256e1fff to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015122849871a825421444a921e03f1256e1fff 2023-10-27 23:59:45,814 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/.tmp/family1/2a857f5bc7dc4d5a9b37582f651bb343 2023-10-27 23:59:45,824 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/.tmp/family1/2a857f5bc7dc4d5a9b37582f651bb343 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/2a857f5bc7dc4d5a9b37582f651bb343 2023-10-27 23:59:45,829 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/2a857f5bc7dc4d5a9b37582f651bb343, entries=1, sequenceid=21, filesize=5.0 K 2023-10-27 23:59:45,830 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bec4bce8b8bdfe22904d678de5936bd8 in 844ms, sequenceid=21, compaction requested=true 2023-10-27 23:59:45,830 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bec4bce8b8bdfe22904d678de5936bd8: 2023-10-27 23:59:45,830 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:45,830 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:45,830 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:45,830 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:45,830 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-10-27 23:59:45,830 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-10-27 23:59:45,859 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:45,859 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:45,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:45,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:45,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:45,859 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:45,860 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:45,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:45,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:45,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:45,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:45,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:45,862 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-10-27 23:59:45,862 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@10e865f2[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:45,862 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-10-27 23:59:45,863 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:45,867 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:45,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:45,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:45,868 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-10-27 23:59:45,868 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-10-27 23:59:45,868 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:45,868 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:45,868 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:45,875 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-10-27 23:59:45,876 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:45,876 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:45,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:45,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:45,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:45,876 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-10-27 23:59:45,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:45,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:45,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:45,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:45,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:45,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:45,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:45,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:45,883 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-10-27 23:59:45,883 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-10-27 23:59:45,883 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:45,884 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:45,884 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:45,884 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:45,892 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:45,892 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:45,893 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:45,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:45,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:45,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:45,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:45,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:45,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:45,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:45,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:45,894 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:45,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:45,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:45,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:45,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:45,898 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:45,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:45,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:45,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:45,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:45,909 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:45,909 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:45,909 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:45,909 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:45,909 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-10-27 23:59:45,909 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:45,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:45,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:45,909 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:45,909 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-10-27 23:59:45,909 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:45,909 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:45,910 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:45,909 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:45,910 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:45,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:45,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:45,910 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:45,910 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,010 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:46,012 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:46,021 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:46,024 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-10-27 23:59:46,024 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,024 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-10-27 23:59:46,024 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:46,024 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-10-27 23:59:46,024 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-10-27 23:59:46,024 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,024 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,034 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:46,034 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:46,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:46,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:46,034 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:46,034 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:46,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,035 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:46,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,035 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,035 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-10-27 23:59:46,035 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,035 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-10-27 23:59:46,035 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:46,036 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-10-27 23:59:46,036 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:46,036 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:46,036 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:46,036 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. started... 2023-10-27 23:59:46,036 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bec4bce8b8bdfe22904d678de5936bd8 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:59:46,451 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512311bebf2ab6b4547b597e3917cee345270 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512311bebf2ab6b4547b597e3917cee345270 2023-10-27 23:59:46,860 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/.tmp/family1/809ff2d87b0b47e283aa7ece594988af 2023-10-27 23:59:46,870 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/.tmp/family1/809ff2d87b0b47e283aa7ece594988af as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/809ff2d87b0b47e283aa7ece594988af 2023-10-27 23:59:46,875 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/809ff2d87b0b47e283aa7ece594988af, entries=1, sequenceid=25, filesize=5.0 K 2023-10-27 23:59:46,876 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bec4bce8b8bdfe22904d678de5936bd8 in 840ms, sequenceid=25, compaction requested=true 2023-10-27 23:59:46,876 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bec4bce8b8bdfe22904d678de5936bd8: 2023-10-27 23:59:46,876 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:46,876 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:46,876 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:46,876 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:46,876 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-10-27 23:59:46,876 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-10-27 23:59:46,917 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:46,917 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:46,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:46,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:46,918 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,918 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:46,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:46,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:46,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:46,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:46,923 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-10-27 23:59:46,923 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@68e8ae92[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:46,923 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-10-27 23:59:46,923 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,934 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,935 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-10-27 23:59:46,935 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-10-27 23:59:46,935 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:46,935 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:46,935 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:46,942 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-10-27 23:59:46,942 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:46,942 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:46,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:46,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:46,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:46,943 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-10-27 23:59:46,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:46,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:46,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:46,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:46,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:46,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:46,951 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-10-27 23:59:46,951 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-10-27 23:59:46,951 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:46,951 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:46,951 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,951 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:46,959 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,959 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:46,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:46,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,959 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:46,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:46,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:46,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:46,960 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:46,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:46,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,962 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:46,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:46,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:46,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:46,995 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:46,995 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,995 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,995 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:46,995 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,995 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-10-27 23:59:46,995 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:46,996 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:46,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:46,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:46,995 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,995 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-10-27 23:59:46,996 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:46,995 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:46,996 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:46,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:46,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:46,997 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:46,997 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:47,098 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:47,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:47,104 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:47,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-10-27 23:59:47,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:47,106 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-10-27 23:59:47,106 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:47,106 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-10-27 23:59:47,106 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-10-27 23:59:47,107 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:47,107 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:47,137 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:47,137 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:47,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:47,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:47,138 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:47,138 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:47,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:47,139 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:47,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:47,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:47,140 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:47,140 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-10-27 23:59:47,140 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:47,141 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-10-27 23:59:47,141 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:47,141 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-10-27 23:59:47,141 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:47,141 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:47,141 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:47,141 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. started... 2023-10-27 23:59:47,142 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bec4bce8b8bdfe22904d678de5936bd8 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:59:47,568 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601018502a928b3fc444bad3ca90abdbceb08 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601018502a928b3fc444bad3ca90abdbceb08 2023-10-27 23:59:47,974 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/.tmp/family1/49fd1b74b03a46d5a9a1b72edf011826 2023-10-27 23:59:47,982 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/.tmp/family1/49fd1b74b03a46d5a9a1b72edf011826 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/49fd1b74b03a46d5a9a1b72edf011826 2023-10-27 23:59:47,989 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/49fd1b74b03a46d5a9a1b72edf011826, entries=1, sequenceid=29, filesize=5.0 K 2023-10-27 23:59:47,990 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bec4bce8b8bdfe22904d678de5936bd8 in 848ms, sequenceid=29, compaction requested=true 2023-10-27 23:59:47,990 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bec4bce8b8bdfe22904d678de5936bd8: 2023-10-27 23:59:47,990 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:47,990 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:47,990 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:47,990 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:47,990 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-10-27 23:59:47,990 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-10-27 23:59:48,026 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:48,026 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:48,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:48,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:48,026 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,026 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:48,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:48,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:48,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:48,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:48,031 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-10-27 23:59:48,031 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5a0ff8ca[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:48,031 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-10-27 23:59:48,032 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,042 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,043 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-10-27 23:59:48,043 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-10-27 23:59:48,043 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:48,043 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:48,043 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:48,051 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-10-27 23:59:48,051 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:48,051 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:48,051 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-10-27 23:59:48,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:48,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:48,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:48,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:48,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:48,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:48,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:48,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:48,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:48,057 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-10-27 23:59:48,057 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-10-27 23:59:48,057 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:48,058 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:48,058 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,058 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:48,067 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,067 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,067 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:48,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:48,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:48,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:48,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:48,069 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:48,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:48,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:48,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,072 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:48,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:48,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:48,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:48,084 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:48,084 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:48,084 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,084 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:48,084 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-10-27 23:59:48,084 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:48,084 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:48,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:48,084 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:48,085 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:48,085 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:48,085 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:48,085 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,086 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,086 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:48,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:48,085 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-10-27 23:59:48,185 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:48,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:48,198 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:48,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-10-27 23:59:48,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,200 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-10-27 23:59:48,200 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:48,200 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-10-27 23:59:48,200 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-10-27 23:59:48,201 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,201 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,209 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:48,209 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:48,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:48,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:48,209 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:48,209 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:48,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,210 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:48,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,210 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,210 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-10-27 23:59:48,210 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:48,210 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-10-27 23:59:48,210 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:48,210 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-10-27 23:59:48,211 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:48,211 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:48,211 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:48,211 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. started... 2023-10-27 23:59:48,211 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bec4bce8b8bdfe22904d678de5936bd8 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:59:48,631 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601035032cf7f5b6e419e8f31a33a0acf90ae to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601035032cf7f5b6e419e8f31a33a0acf90ae 2023-10-27 23:59:49,039 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/.tmp/family1/70af4aab19104841822823b948f3adb6 2023-10-27 23:59:49,048 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/.tmp/family1/70af4aab19104841822823b948f3adb6 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/70af4aab19104841822823b948f3adb6 2023-10-27 23:59:49,056 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/70af4aab19104841822823b948f3adb6, entries=1, sequenceid=33, filesize=5.0 K 2023-10-27 23:59:49,057 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bec4bce8b8bdfe22904d678de5936bd8 in 846ms, sequenceid=33, compaction requested=true 2023-10-27 23:59:49,057 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bec4bce8b8bdfe22904d678de5936bd8: 2023-10-27 23:59:49,057 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:49,057 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:49,057 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:49,057 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:49,057 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-10-27 23:59:49,057 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-10-27 23:59:49,092 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:49,092 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:49,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:49,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:49,093 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,093 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:49,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:49,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:49,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:49,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:49,095 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-10-27 23:59:49,095 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@11a30160[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:49,095 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-10-27 23:59:49,095 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,100 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,101 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-10-27 23:59:49,101 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-10-27 23:59:49,101 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:49,101 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:49,101 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:49,109 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-10-27 23:59:49,109 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:49,109 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:49,109 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-10-27 23:59:49,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:49,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:49,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:49,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:49,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:49,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:49,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:49,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:49,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:49,117 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-10-27 23:59:49,117 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-10-27 23:59:49,117 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:49,117 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:49,117 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,117 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:49,126 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,126 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,126 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:49,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:49,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:49,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:49,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:49,127 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:49,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:49,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:49,130 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:49,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:49,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:49,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:49,142 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:49,142 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:49,142 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:49,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:49,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:49,142 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,142 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-10-27 23:59:49,142 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:49,142 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:49,142 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,142 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-10-27 23:59:49,143 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:49,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:49,143 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:49,142 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:49,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:49,143 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,143 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,143 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,243 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:49,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:49,249 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:49,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-10-27 23:59:49,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,251 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-10-27 23:59:49,251 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:49,251 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-10-27 23:59:49,252 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-10-27 23:59:49,252 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,252 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,259 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:49,259 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:49,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:49,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:49,259 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:49,259 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:49,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,260 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:49,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,260 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,260 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-10-27 23:59:49,260 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:49,260 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-10-27 23:59:49,260 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:49,261 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-10-27 23:59:49,261 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:49,261 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:49,261 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:49,261 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. started... 2023-10-27 23:59:49,261 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bec4bce8b8bdfe22904d678de5936bd8 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:59:49,680 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907e3cc2718e3764224823896c41afa91f6 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20150907e3cc2718e3764224823896c41afa91f6 2023-10-27 23:59:50,087 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/.tmp/family1/dea0e953947d4c418d84321ebfbb65c4 2023-10-27 23:59:50,098 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/.tmp/family1/dea0e953947d4c418d84321ebfbb65c4 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/dea0e953947d4c418d84321ebfbb65c4 2023-10-27 23:59:50,103 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/dea0e953947d4c418d84321ebfbb65c4, entries=1, sequenceid=37, filesize=5.0 K 2023-10-27 23:59:50,103 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bec4bce8b8bdfe22904d678de5936bd8 in 842ms, sequenceid=37, compaction requested=true 2023-10-27 23:59:50,103 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bec4bce8b8bdfe22904d678de5936bd8: 2023-10-27 23:59:50,103 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:50,104 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:50,104 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:50,104 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:50,104 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-10-27 23:59:50,104 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-10-27 23:59:50,134 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:50,134 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:50,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:50,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:50,134 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,134 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:50,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:50,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:50,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:50,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:50,136 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-10-27 23:59:50,136 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@42b73153[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:50,136 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-10-27 23:59:50,136 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,142 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,142 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-10-27 23:59:50,142 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-10-27 23:59:50,142 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:50,142 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:50,142 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:50,150 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-10-27 23:59:50,150 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:50,150 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:50,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:50,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:50,150 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-10-27 23:59:50,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:50,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:50,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:50,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:50,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:50,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:50,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:50,153 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-10-27 23:59:50,153 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-10-27 23:59:50,153 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:50,153 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:50,153 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,153 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:50,159 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,159 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:50,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:50,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,159 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:50,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:50,159 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:50,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:50,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:50,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:50,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:50,160 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:50,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:50,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:50,167 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:50,167 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:50,167 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:50,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:50,167 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-10-27 23:59:50,167 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:50,167 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,167 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:50,167 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:50,167 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:50,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:50,168 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,168 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:50,168 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:50,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:50,168 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,168 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,268 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:50,269 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:50,274 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-27 23:59:50,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-10-27 23:59:50,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,277 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-10-27 23:59:50,277 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:50,277 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-10-27 23:59:50,277 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-10-27 23:59:50,278 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,278 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,284 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:50,284 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:50,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:50,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:50,285 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:50,285 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-27 23:59:50,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,286 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-27 23:59:50,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,287 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,287 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-10-27 23:59:50,287 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:50,288 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-10-27 23:59:50,288 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-27 23:59:50,288 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-10-27 23:59:50,288 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-27 23:59:50,288 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-27 23:59:50,288 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:50,288 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. started... 2023-10-27 23:59:50,288 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bec4bce8b8bdfe22904d678de5936bd8 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-27 23:59:50,724 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-10-27 23:59:50,724 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x162e014f to 127.0.0.1:57369 2023-10-27 23:59:50,724 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-10-27 23:59:50,727 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7554ee49 to 127.0.0.1:57369 2023-10-27 23:59:50,727 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-10-27 23:59:50,729 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-10-27 23:59:50,729 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x532035c1 to 127.0.0.1:57369 2023-10-27 23:59:50,729 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-10-27 23:59:50,729 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-10-27 23:59:50,730 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2470ae6f to 127.0.0.1:57369 2023-10-27 23:59:50,730 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-10-27 23:59:50,730 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-10-27 23:59:50,730 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0415b3ca to 127.0.0.1:57369 2023-10-27 23:59:50,730 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-10-27 23:59:50,736 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511204878dd8d67684fbbb5d9e462a9a22197 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511204878dd8d67684fbbb5d9e462a9a22197 2023-10-27 23:59:51,143 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/.tmp/family1/c14edd2e728545dda58ccb4c9653c52e 2023-10-27 23:59:51,156 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/.tmp/family1/c14edd2e728545dda58ccb4c9653c52e as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/c14edd2e728545dda58ccb4c9653c52e 2023-10-27 23:59:51,162 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/c14edd2e728545dda58ccb4c9653c52e, entries=1, sequenceid=41, filesize=5.0 K 2023-10-27 23:59:51,163 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bec4bce8b8bdfe22904d678de5936bd8 in 875ms, sequenceid=41, compaction requested=true 2023-10-27 23:59:51,163 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bec4bce8b8bdfe22904d678de5936bd8: 2023-10-27 23:59:51,163 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:51,163 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-27 23:59:51,163 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-27 23:59:51,163 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:51,164 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-10-27 23:59:51,164 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-10-27 23:59:51,192 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:51,192 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:51,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:51,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:51,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:51,193 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:51,193 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-27 23:59:51,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:51,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:51,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:51,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:51,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:51,198 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-10-27 23:59:51,198 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6f868ba9[Count = 0] remaining members to acquire global barrier 2023-10-27 23:59:51,198 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-10-27 23:59:51,198 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:51,209 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:51,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:51,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:51,209 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-10-27 23:59:51,210 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-10-27 23:59:51,210 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-27 23:59:51,210 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:51,210 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-27 23:59:51,217 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:51,217 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-10-27 23:59:51,218 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:51,218 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-10-27 23:59:51,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:51,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:51,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:51,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:51,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:51,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:51,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:51,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:51,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:51,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:51,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-27 23:59:51,228 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-10-27 23:59:51,228 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-10-27 23:59:51,228 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-27 23:59:51,228 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-27 23:59:51,229 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:51,229 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-27 23:59:51,240 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:51,241 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:51,241 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:51,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:51,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:51,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:51,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-27 23:59:51,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-27 23:59:51,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:51,242 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:51,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:51,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-27 23:59:51,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:51,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:51,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-27 23:59:51,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:51,245 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:51,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:51,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-27 23:59:51,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:51,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:51,259 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:51,259 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-27 23:59:51,259 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:51,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-27 23:59:51,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-27 23:59:51,259 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-10-27 23:59:51,259 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-27 23:59:51,259 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-27 23:59:51,259 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:51,259 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-27 23:59:51,260 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-10-27 23:59:51,260 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-27 23:59:51,260 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:51,260 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-10-27 23:59:51,260 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:51,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-27 23:59:51,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-27 23:59:51,261 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:51,261 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:51,360 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-27 23:59:51,361 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-27 23:59:51,362 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36413] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:51,364 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] procedure2.ProcedureExecutor(1028): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-10-27 23:59:51,364 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-10-27 23:59:51,365 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-27 23:59:51,365 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698451791365 2023-10-27 23:59:51,365 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-10-27 23:59:51,366 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-10-27 23:59:51,366 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-10-27 23:59:51,368 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] zookeeper.ReadOnlyZKClient(139): Connect 0x07006e84 to 127.0.0.1:57369 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-10-27 23:59:51,376 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7ef83d73, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-27 23:59:51,376 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-10-27 23:59:52,256 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601038c63695b0380489485476383bbeb952b to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601038c63695b0380489485476383bbeb952b 2023-10-27 23:59:52,257 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-10-27 23:59:52,268 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-10-27 23:59:52,269 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:57218, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-10-27 23:59:52,275 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-10-27 23:59:52,276 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:43294, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-10-27 23:59:52,285 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/c8b5e6d6bb494404aa6785ecf02f876c first=Optional[r3] last=Optional[r6] 2023-10-27 23:59:52,286 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/c8b5e6d6bb494404aa6785ecf02f876c}] 2023-10-27 23:59:52,290 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/c8b5e6d6bb494404aa6785ecf02f876c for inclusion in bec4bce8b8bdfe22904d678de5936bd8/family1 2023-10-27 23:59:52,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-10-27 23:59:52,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(777): Region bounds: first= last= 2023-10-27 23:59:52,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for bec4bce8b8bdfe22904d678de5936bd8: 2023-10-27 23:59:52,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/c8b5e6d6bb494404aa6785ecf02f876c to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__9atei0jk7usbscdvhso9ghbtmr93603n4v1ubo6d3qc48f8agp6jegq8vaif86fa/family1/c8b5e6d6bb494404aa6785ecf02f876c 2023-10-27 23:59:52,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__9atei0jk7usbscdvhso9ghbtmr93603n4v1ubo6d3qc48f8agp6jegq8vaif86fa/family1/c8b5e6d6bb494404aa6785ecf02f876c as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/1935c5bebac3466dba96748c50ac49c3_SeqId_44_ 2023-10-27 23:59:52,296 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__9atei0jk7usbscdvhso9ghbtmr93603n4v1ubo6d3qc48f8agp6jegq8vaif86fa/family1/c8b5e6d6bb494404aa6785ecf02f876c into bec4bce8b8bdfe22904d678de5936bd8/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/1935c5bebac3466dba96748c50ac49c3_SeqId_44_ - updating store file list. 2023-10-27 23:59:52,300 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/1935c5bebac3466dba96748c50ac49c3_SeqId_44_ into bec4bce8b8bdfe22904d678de5936bd8/family1 2023-10-27 23:59:52,300 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__9atei0jk7usbscdvhso9ghbtmr93603n4v1ubo6d3qc48f8agp6jegq8vaif86fa/family1/c8b5e6d6bb494404aa6785ecf02f876c into bec4bce8b8bdfe22904d678de5936bd8/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/1935c5bebac3466dba96748c50ac49c3_SeqId_44_) 2023-10-27 23:59:52,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__9atei0jk7usbscdvhso9ghbtmr93603n4v1ubo6d3qc48f8agp6jegq8vaif86fa/family1/c8b5e6d6bb494404aa6785ecf02f876c 2023-10-27 23:59:52,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-10-27 23:59:52,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-10-27 23:59:52,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(1912): bec4bce8b8bdfe22904d678de5936bd8/family1 is initiating minor compaction 2023-10-27 23:59:52,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@178f9875; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-27 23:59:52,303 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of bec4bce8b8bdfe22904d678de5936bd8/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:52,303 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/a17f44acaec44421b1db8fb7e8bcba36, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/cf16caabeddf43df9b074645f7a54fdb, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/1fe844c9466a4aa4b7abab315c09a72a, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/2a857f5bc7dc4d5a9b37582f651bb343, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/809ff2d87b0b47e283aa7ece594988af, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/49fd1b74b03a46d5a9a1b72edf011826, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/70af4aab19104841822823b948f3adb6, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/dea0e953947d4c418d84321ebfbb65c4, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/c14edd2e728545dda58ccb4c9653c52e, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/1935c5bebac3466dba96748c50ac49c3_SeqId_44_] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/.tmp, totalSize=50.3 K 2023-10-27 23:59:52,303 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting a17f44acaec44421b1db8fb7e8bcba36, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-10-27 23:59:52,304 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting cf16caabeddf43df9b074645f7a54fdb, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-10-27 23:59:52,304 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 1fe844c9466a4aa4b7abab315c09a72a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-10-27 23:59:52,304 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 2a857f5bc7dc4d5a9b37582f651bb343, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-10-27 23:59:52,304 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 809ff2d87b0b47e283aa7ece594988af, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-10-27 23:59:52,304 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 49fd1b74b03a46d5a9a1b72edf011826, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-10-27 23:59:52,304 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 70af4aab19104841822823b948f3adb6, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-10-27 23:59:52,304 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting dea0e953947d4c418d84321ebfbb65c4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-10-27 23:59:52,304 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting c14edd2e728545dda58ccb4c9653c52e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-10-27 23:59:52,304 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 1935c5bebac3466dba96748c50ac49c3_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-10-27 23:59:52,305 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-10-27 23:59:52,306 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:59:52,307 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015122849871a825421444a921e03f1256e1fff to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015122849871a825421444a921e03f1256e1fff 2023-10-27 23:59:52,308 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512311bebf2ab6b4547b597e3917cee345270 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512311bebf2ab6b4547b597e3917cee345270 2023-10-27 23:59:52,309 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601018502a928b3fc444bad3ca90abdbceb08 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601018502a928b3fc444bad3ca90abdbceb08 2023-10-27 23:59:52,310 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601035032cf7f5b6e419e8f31a33a0acf90ae to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601035032cf7f5b6e419e8f31a33a0acf90ae 2023-10-27 23:59:52,310 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-10-27 23:59:52,310 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-10-27 23:59:52,311 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-10-27 23:59:52,311 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-10-27 23:59:52,311 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-10-27 23:59:52,311 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-10-27 23:59:52,311 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-10-27 23:59:52,311 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-10-27 23:59:52,311 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-10-27 23:59:52,311 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-10-27 23:59:52,355 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] throttle.PressureAwareThroughputController(145): bec4bce8b8bdfe22904d678de5936bd8#family1#compaction#73 average throughput is 0.20 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 73.08 MB/second 2023-10-27 23:59:52,785 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/.tmp/family1/4b3431b727384a79abbc22ff373cab72 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/4b3431b727384a79abbc22ff373cab72 2023-10-27 23:59:52,791 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in bec4bce8b8bdfe22904d678de5936bd8/family1 of bec4bce8b8bdfe22904d678de5936bd8 into 4b3431b727384a79abbc22ff373cab72(size=6.8 K), total size for store is 197.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-10-27 23:59:52,791 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for bec4bce8b8bdfe22904d678de5936bd8: 2023-10-27 23:59:52,791 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1698451192303; duration=0sec 2023-10-27 23:59:52,791 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-27 23:59:52,840 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/a17f44acaec44421b1db8fb7e8bcba36, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/cf16caabeddf43df9b074645f7a54fdb, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/1fe844c9466a4aa4b7abab315c09a72a, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/2a857f5bc7dc4d5a9b37582f651bb343, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/809ff2d87b0b47e283aa7ece594988af, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/49fd1b74b03a46d5a9a1b72edf011826, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/70af4aab19104841822823b948f3adb6, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/dea0e953947d4c418d84321ebfbb65c4, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/c14edd2e728545dda58ccb4c9653c52e, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/1935c5bebac3466dba96748c50ac49c3_SeqId_44_] to archive 2023-10-27 23:59:52,841 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:59:52,843 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/a17f44acaec44421b1db8fb7e8bcba36 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/a17f44acaec44421b1db8fb7e8bcba36 2023-10-27 23:59:52,845 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/cf16caabeddf43df9b074645f7a54fdb to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/cf16caabeddf43df9b074645f7a54fdb 2023-10-27 23:59:52,847 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/1fe844c9466a4aa4b7abab315c09a72a to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/1fe844c9466a4aa4b7abab315c09a72a 2023-10-27 23:59:52,849 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/2a857f5bc7dc4d5a9b37582f651bb343 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/2a857f5bc7dc4d5a9b37582f651bb343 2023-10-27 23:59:52,851 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/809ff2d87b0b47e283aa7ece594988af to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/809ff2d87b0b47e283aa7ece594988af 2023-10-27 23:59:52,853 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/49fd1b74b03a46d5a9a1b72edf011826 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/49fd1b74b03a46d5a9a1b72edf011826 2023-10-27 23:59:52,854 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/70af4aab19104841822823b948f3adb6 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/70af4aab19104841822823b948f3adb6 2023-10-27 23:59:52,856 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/dea0e953947d4c418d84321ebfbb65c4 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/dea0e953947d4c418d84321ebfbb65c4 2023-10-27 23:59:52,858 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/c14edd2e728545dda58ccb4c9653c52e to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/c14edd2e728545dda58ccb4c9653c52e 2023-10-27 23:59:52,859 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/1935c5bebac3466dba96748c50ac49c3_SeqId_44_ to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/1935c5bebac3466dba96748c50ac49c3_SeqId_44_ 2023-10-27 23:59:53,194 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512056f6ea555f1ac41afb5a6eadd58cb6467 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512056f6ea555f1ac41afb5a6eadd58cb6467 2023-10-27 23:59:53,196 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-10-27 23:59:53,222 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/39da480fd739462a93226dbe9002b437 first=Optional[r00] last=Optional[r2] 2023-10-27 23:59:53,223 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/39da480fd739462a93226dbe9002b437}] 2023-10-27 23:59:53,225 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/39da480fd739462a93226dbe9002b437 for inclusion in bec4bce8b8bdfe22904d678de5936bd8/family1 2023-10-27 23:59:53,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-10-27 23:59:53,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(777): Region bounds: first= last= 2023-10-27 23:59:53,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for bec4bce8b8bdfe22904d678de5936bd8: 2023-10-27 23:59:53,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/39da480fd739462a93226dbe9002b437 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__3m6op1uvlb3sc42sv7d74pk03ntgk6g3hkgiq6f6l1jne4gv6q99fguj9e3jhlut/family1/39da480fd739462a93226dbe9002b437 2023-10-27 23:59:53,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__3m6op1uvlb3sc42sv7d74pk03ntgk6g3hkgiq6f6l1jne4gv6q99fguj9e3jhlut/family1/39da480fd739462a93226dbe9002b437 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/5b8304cc39054f658db6e3891c541fae_SeqId_47_ 2023-10-27 23:59:53,234 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__3m6op1uvlb3sc42sv7d74pk03ntgk6g3hkgiq6f6l1jne4gv6q99fguj9e3jhlut/family1/39da480fd739462a93226dbe9002b437 into bec4bce8b8bdfe22904d678de5936bd8/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/5b8304cc39054f658db6e3891c541fae_SeqId_47_ - updating store file list. 2023-10-27 23:59:53,241 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/5b8304cc39054f658db6e3891c541fae_SeqId_47_ into bec4bce8b8bdfe22904d678de5936bd8/family1 2023-10-27 23:59:53,241 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__3m6op1uvlb3sc42sv7d74pk03ntgk6g3hkgiq6f6l1jne4gv6q99fguj9e3jhlut/family1/39da480fd739462a93226dbe9002b437 into bec4bce8b8bdfe22904d678de5936bd8/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/5b8304cc39054f658db6e3891c541fae_SeqId_47_) 2023-10-27 23:59:53,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__3m6op1uvlb3sc42sv7d74pk03ntgk6g3hkgiq6f6l1jne4gv6q99fguj9e3jhlut/family1/39da480fd739462a93226dbe9002b437 2023-10-27 23:59:53,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-10-27 23:59:53,243 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-10-27 23:59:53,243 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(1912): bec4bce8b8bdfe22904d678de5936bd8/family1 is initiating minor compaction (all files) 2023-10-27 23:59:53,243 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@12f430eb; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-10-27 23:59:53,243 INFO [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.HRegion(2259): Starting compaction of bec4bce8b8bdfe22904d678de5936bd8/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:53,243 INFO [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/464183528ba54a72980a4158ab1c2468, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/4b3431b727384a79abbc22ff373cab72, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/5b8304cc39054f658db6e3891c541fae_SeqId_47_] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/.tmp, totalSize=388.1 K 2023-10-27 23:59:53,244 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] compactions.Compactor(207): Compacting 464183528ba54a72980a4158ab1c2468, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878821089 2023-10-27 23:59:53,244 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] compactions.Compactor(207): Compacting 4b3431b727384a79abbc22ff373cab72, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621221089 2023-10-27 23:59:53,245 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] compactions.Compactor(207): Compacting 5b8304cc39054f658db6e3891c541fae_SeqId_47_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878821089 2023-10-27 23:59:53,245 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-10-27 23:59:53,246 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:59:53,247 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130be3fc91ca8b145079380a7623cce125f to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130be3fc91ca8b145079380a7623cce125f 2023-10-27 23:59:53,248 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512018db9338f82964fbdbd524281f93f200a to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512018db9338f82964fbdbd524281f93f200a 2023-10-27 23:59:53,249 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512058f06e4a26a454bc0b4f55be8aa56e0aa to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512058f06e4a26a454bc0b4f55be8aa56e0aa 2023-10-27 23:59:53,249 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-10-27 23:59:53,249 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-10-27 23:59:53,250 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-10-27 23:59:53,250 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-10-27 23:59:53,250 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-27 23:59:53,251 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-10-27 23:59:53,252 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 1.8880 sec 2023-10-27 23:59:53,262 INFO [RS:0;jenkins-hbase16:43469-longCompactions-0] throttle.PressureAwareThroughputController(145): bec4bce8b8bdfe22904d678de5936bd8#family1#compaction#74 average throughput is 22.45 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 73.08 MB/second 2023-10-27 23:59:53,326 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.HMaster$12(2454): Client=jenkins//188.40.62.62 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-10-27 23:59:53,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure2.ProcedureExecutor(1028): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:53,739 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-10-27 23:59:53,742 DEBUG [PEWorker-4] util.FSTableDescriptors(519): Deleted hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-10-27 23:59:53,742 INFO [PEWorker-4] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-10-27 23:59:53,747 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-10-27 23:59:53,751 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=bec4bce8b8bdfe22904d678de5936bd8, REOPEN/MOVE}] 2023-10-27 23:59:53,752 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=bec4bce8b8bdfe22904d678de5936bd8, REOPEN/MOVE 2023-10-27 23:59:53,752 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=bec4bce8b8bdfe22904d678de5936bd8, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:53,753 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1698451193752"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451193752"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451193752"}]},"ts":"1698451193752"} 2023-10-27 23:59:53,754 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure bec4bce8b8bdfe22904d678de5936bd8, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-27 23:59:53,906 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:53,906 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(111): Close bec4bce8b8bdfe22904d678de5936bd8 2023-10-27 23:59:53,907 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing bec4bce8b8bdfe22904d678de5936bd8, disabling compactions & flushes 2023-10-27 23:59:53,907 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:54,497 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/.tmp/family1/e5117055d61b47679966a5d5f4823d34 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/e5117055d61b47679966a5d5f4823d34 2023-10-27 23:59:54,504 INFO [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in bec4bce8b8bdfe22904d678de5936bd8/family1 of bec4bce8b8bdfe22904d678de5936bd8 into e5117055d61b47679966a5d5f4823d34(size=192.0 K), total size for store is 192.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-10-27 23:59:54,504 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for bec4bce8b8bdfe22904d678de5936bd8: 2023-10-27 23:59:54,504 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:54,504 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:54,504 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. after waiting 0 ms 2023-10-27 23:59:54,504 INFO [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8., storeName=family1, fileCount=3, fileSize=388.1 K (190.5 K, 6.8 K, 190.9 K), priority=2, time=1698451193243; duration=1sec 2023-10-27 23:59:54,504 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:54,504 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-27 23:59:54,508 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/464183528ba54a72980a4158ab1c2468, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/4b3431b727384a79abbc22ff373cab72, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/5b8304cc39054f658db6e3891c541fae_SeqId_47_] to archive 2023-10-27 23:59:54,509 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:59:54,512 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/464183528ba54a72980a4158ab1c2468 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/464183528ba54a72980a4158ab1c2468 2023-10-27 23:59:54,513 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/4b3431b727384a79abbc22ff373cab72 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/4b3431b727384a79abbc22ff373cab72 2023-10-27 23:59:54,515 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/5b8304cc39054f658db6e3891c541fae_SeqId_47_ to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/5b8304cc39054f658db6e3891c541fae_SeqId_47_ 2023-10-27 23:59:54,520 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-10-27 23:59:54,521 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:54,521 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for bec4bce8b8bdfe22904d678de5936bd8: 2023-10-27 23:59:54,521 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(3510): Not adding moved region record: bec4bce8b8bdfe22904d678de5936bd8 to self. 2023-10-27 23:59:54,523 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(149): Closed bec4bce8b8bdfe22904d678de5936bd8 2023-10-27 23:59:54,524 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=bec4bce8b8bdfe22904d678de5936bd8, regionState=CLOSED 2023-10-27 23:59:54,524 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1698451194524"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451194524"}]},"ts":"1698451194524"} 2023-10-27 23:59:54,527 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=62, resume processing ppid=61 2023-10-27 23:59:54,527 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure bec4bce8b8bdfe22904d678de5936bd8, server=jenkins-hbase16.apache.org,43469,1698451096272 in 772 msec 2023-10-27 23:59:54,528 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=bec4bce8b8bdfe22904d678de5936bd8, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase16.apache.org,43469,1698451096272; forceNewPlan=false, retain=true 2023-10-27 23:59:54,679 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=bec4bce8b8bdfe22904d678de5936bd8, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:54,679 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1698451194679"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451194679"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451194679"}]},"ts":"1698451194679"} 2023-10-27 23:59:54,682 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure bec4bce8b8bdfe22904d678de5936bd8, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-27 23:59:54,835 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:54,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:54,840 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => bec4bce8b8bdfe22904d678de5936bd8, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8.', STARTKEY => '', ENDKEY => ''} 2023-10-27 23:59:54,840 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily bec4bce8b8bdfe22904d678de5936bd8 2023-10-27 23:59:54,840 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:59:54,841 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for bec4bce8b8bdfe22904d678de5936bd8 2023-10-27 23:59:54,841 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for bec4bce8b8bdfe22904d678de5936bd8 2023-10-27 23:59:54,843 INFO [StoreOpener-bec4bce8b8bdfe22904d678de5936bd8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region bec4bce8b8bdfe22904d678de5936bd8 2023-10-27 23:59:54,845 DEBUG [StoreOpener-bec4bce8b8bdfe22904d678de5936bd8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1 2023-10-27 23:59:54,845 DEBUG [StoreOpener-bec4bce8b8bdfe22904d678de5936bd8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1 2023-10-27 23:59:54,846 INFO [StoreOpener-bec4bce8b8bdfe22904d678de5936bd8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bec4bce8b8bdfe22904d678de5936bd8 columnFamilyName family1 2023-10-27 23:59:54,851 DEBUG [StoreOpener-bec4bce8b8bdfe22904d678de5936bd8-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/e5117055d61b47679966a5d5f4823d34 2023-10-27 23:59:54,851 INFO [StoreOpener-bec4bce8b8bdfe22904d678de5936bd8-1] regionserver.HStore(310): Store=bec4bce8b8bdfe22904d678de5936bd8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-27 23:59:54,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8 2023-10-27 23:59:54,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8 2023-10-27 23:59:54,855 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for bec4bce8b8bdfe22904d678de5936bd8 2023-10-27 23:59:54,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened bec4bce8b8bdfe22904d678de5936bd8; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74426615, jitterRate=0.10904298722743988}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-10-27 23:59:54,856 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for bec4bce8b8bdfe22904d678de5936bd8: 2023-10-27 23:59:54,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8., pid=63, masterSystemTime=1698451194835 2023-10-27 23:59:54,858 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:54,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:54,858 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=bec4bce8b8bdfe22904d678de5936bd8, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:54,859 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1698451194858"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451194858"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451194858"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451194858"}]},"ts":"1698451194858"} 2023-10-27 23:59:54,861 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=63, resume processing ppid=61 2023-10-27 23:59:54,861 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure bec4bce8b8bdfe22904d678de5936bd8, server=jenkins-hbase16.apache.org,43469,1698451096272 in 178 msec 2023-10-27 23:59:54,863 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=61, resume processing ppid=60 2023-10-27 23:59:54,863 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=bec4bce8b8bdfe22904d678de5936bd8, REOPEN/MOVE in 1.1100 sec 2023-10-27 23:59:54,865 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=60, resume processing ppid=59 2023-10-27 23:59:54,865 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1170 sec 2023-10-27 23:59:54,866 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.5400 sec 2023-10-27 23:59:54,867 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=59 2023-10-27 23:59:54,868 INFO [Listener at localhost/40295] compactions.TestMobCompactor(278): alter status finished 2023-10-27 23:59:54,869 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36413] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:54,870 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] procedure2.ProcedureExecutor(1028): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-10-27 23:59:54,871 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-10-27 23:59:54,871 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-27 23:59:54,871 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698451794871 2023-10-27 23:59:54,872 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-10-27 23:59:54,873 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-10-27 23:59:54,873 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-10-27 23:59:54,874 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] zookeeper.ReadOnlyZKClient(139): Connect 0x0c6d1a6f to 127.0.0.1:57369 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-10-27 23:59:54,910 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@54f2b198, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-27 23:59:54,911 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-10-27 23:59:54,911 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-10-27 23:59:54,911 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-10-27 23:59:54,911 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-10-27 23:59:54,911 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-10-27 23:59:55,751 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128b2e7c7f64d0d4d74bb52e8b930a7cf6b to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128b2e7c7f64d0d4d74bb52e8b930a7cf6b 2023-10-27 23:59:55,753 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-10-27 23:59:55,768 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-10-27 23:59:55,769 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:50654, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-10-27 23:59:55,775 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-10-27 23:59:55,776 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:57892, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-10-27 23:59:55,786 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/15768212396e4036b5dfcf0bf4b57160 first=Optional[r05] last=Optional[r8] 2023-10-27 23:59:55,787 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=51 for row with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/15768212396e4036b5dfcf0bf4b57160}] 2023-10-27 23:59:55,791 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/15768212396e4036b5dfcf0bf4b57160 for inclusion in bec4bce8b8bdfe22904d678de5936bd8/family1 2023-10-27 23:59:55,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-10-27 23:59:55,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(777): Region bounds: first= last= 2023-10-27 23:59:55,795 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for bec4bce8b8bdfe22904d678de5936bd8: 2023-10-27 23:59:55,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/15768212396e4036b5dfcf0bf4b57160 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__8n0fti0ipdndoe3cioo50b05kp2bt8j87hatacbo2173e9dqip3lbup7fpik299k/family1/15768212396e4036b5dfcf0bf4b57160 2023-10-27 23:59:55,798 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__8n0fti0ipdndoe3cioo50b05kp2bt8j87hatacbo2173e9dqip3lbup7fpik299k/family1/15768212396e4036b5dfcf0bf4b57160 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/c845a6e90a6f42fd90195f6c55728a69_SeqId_53_ 2023-10-27 23:59:55,798 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__8n0fti0ipdndoe3cioo50b05kp2bt8j87hatacbo2173e9dqip3lbup7fpik299k/family1/15768212396e4036b5dfcf0bf4b57160 into bec4bce8b8bdfe22904d678de5936bd8/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/c845a6e90a6f42fd90195f6c55728a69_SeqId_53_ - updating store file list. 2023-10-27 23:59:55,803 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/c845a6e90a6f42fd90195f6c55728a69_SeqId_53_ into bec4bce8b8bdfe22904d678de5936bd8/family1 2023-10-27 23:59:55,803 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__8n0fti0ipdndoe3cioo50b05kp2bt8j87hatacbo2173e9dqip3lbup7fpik299k/family1/15768212396e4036b5dfcf0bf4b57160 into bec4bce8b8bdfe22904d678de5936bd8/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/c845a6e90a6f42fd90195f6c55728a69_SeqId_53_) 2023-10-27 23:59:55,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__8n0fti0ipdndoe3cioo50b05kp2bt8j87hatacbo2173e9dqip3lbup7fpik299k/family1/15768212396e4036b5dfcf0bf4b57160 2023-10-27 23:59:55,806 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-10-27 23:59:55,807 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-27 23:59:55,808 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511204878dd8d67684fbbb5d9e462a9a22197 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511204878dd8d67684fbbb5d9e462a9a22197 2023-10-27 23:59:55,809 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128bb9aa9dd76ac44338809008402709ffe to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128bb9aa9dd76ac44338809008402709ffe 2023-10-27 23:59:55,809 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-10-27 23:59:55,809 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-10-27 23:59:55,809 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-10-27 23:59:55,809 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-10-27 23:59:55,809 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-10-27 23:59:55,809 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-10-27 23:59:55,810 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-27 23:59:55,811 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-10-27 23:59:55,812 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 941 msec 2023-10-27 23:59:55,868 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.HMaster$12(2454): Client=jenkins//188.40.62.62 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-10-27 23:59:55,869 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure2.ProcedureExecutor(1028): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:56,281 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-10-27 23:59:56,284 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-10-27 23:59:56,284 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-10-27 23:59:56,289 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-10-27 23:59:56,292 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=bec4bce8b8bdfe22904d678de5936bd8, REOPEN/MOVE}] 2023-10-27 23:59:56,293 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=bec4bce8b8bdfe22904d678de5936bd8, REOPEN/MOVE 2023-10-27 23:59:56,294 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=bec4bce8b8bdfe22904d678de5936bd8, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:56,294 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1698451196294"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451196294"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451196294"}]},"ts":"1698451196294"} 2023-10-27 23:59:56,295 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure bec4bce8b8bdfe22904d678de5936bd8, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-27 23:59:56,447 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:56,448 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(111): Close bec4bce8b8bdfe22904d678de5936bd8 2023-10-27 23:59:56,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing bec4bce8b8bdfe22904d678de5936bd8, disabling compactions & flushes 2023-10-27 23:59:56,448 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:56,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:56,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. after waiting 0 ms 2023-10-27 23:59:56,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:56,457 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-10-27 23:59:56,458 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:56,458 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for bec4bce8b8bdfe22904d678de5936bd8: 2023-10-27 23:59:56,458 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(3510): Not adding moved region record: bec4bce8b8bdfe22904d678de5936bd8 to self. 2023-10-27 23:59:56,460 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(149): Closed bec4bce8b8bdfe22904d678de5936bd8 2023-10-27 23:59:56,460 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=bec4bce8b8bdfe22904d678de5936bd8, regionState=CLOSED 2023-10-27 23:59:56,460 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1698451196460"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451196460"}]},"ts":"1698451196460"} 2023-10-27 23:59:56,463 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=68, resume processing ppid=67 2023-10-27 23:59:56,463 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure bec4bce8b8bdfe22904d678de5936bd8, server=jenkins-hbase16.apache.org,43469,1698451096272 in 167 msec 2023-10-27 23:59:56,464 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=bec4bce8b8bdfe22904d678de5936bd8, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase16.apache.org,43469,1698451096272; forceNewPlan=false, retain=true 2023-10-27 23:59:56,615 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=bec4bce8b8bdfe22904d678de5936bd8, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:56,615 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1698451196614"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451196614"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451196614"}]},"ts":"1698451196614"} 2023-10-27 23:59:56,620 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure bec4bce8b8bdfe22904d678de5936bd8, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-27 23:59:56,772 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:56,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:56,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => bec4bce8b8bdfe22904d678de5936bd8, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8.', STARTKEY => '', ENDKEY => ''} 2023-10-27 23:59:56,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily bec4bce8b8bdfe22904d678de5936bd8 2023-10-27 23:59:56,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:59:56,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for bec4bce8b8bdfe22904d678de5936bd8 2023-10-27 23:59:56,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for bec4bce8b8bdfe22904d678de5936bd8 2023-10-27 23:59:56,782 INFO [StoreOpener-bec4bce8b8bdfe22904d678de5936bd8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region bec4bce8b8bdfe22904d678de5936bd8 2023-10-27 23:59:56,784 DEBUG [StoreOpener-bec4bce8b8bdfe22904d678de5936bd8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1 2023-10-27 23:59:56,784 DEBUG [StoreOpener-bec4bce8b8bdfe22904d678de5936bd8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1 2023-10-27 23:59:56,784 INFO [StoreOpener-bec4bce8b8bdfe22904d678de5936bd8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bec4bce8b8bdfe22904d678de5936bd8 columnFamilyName family1 2023-10-27 23:59:56,790 DEBUG [StoreOpener-bec4bce8b8bdfe22904d678de5936bd8-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/c845a6e90a6f42fd90195f6c55728a69_SeqId_53_ 2023-10-27 23:59:56,794 DEBUG [StoreOpener-bec4bce8b8bdfe22904d678de5936bd8-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/family1/e5117055d61b47679966a5d5f4823d34 2023-10-27 23:59:56,795 INFO [StoreOpener-bec4bce8b8bdfe22904d678de5936bd8-1] regionserver.HStore(310): Store=bec4bce8b8bdfe22904d678de5936bd8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-27 23:59:56,796 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8 2023-10-27 23:59:56,798 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8 2023-10-27 23:59:56,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for bec4bce8b8bdfe22904d678de5936bd8 2023-10-27 23:59:56,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened bec4bce8b8bdfe22904d678de5936bd8; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59964580, jitterRate=-0.10645812749862671}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-10-27 23:59:56,802 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for bec4bce8b8bdfe22904d678de5936bd8: 2023-10-27 23:59:56,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8., pid=69, masterSystemTime=1698451196772 2023-10-27 23:59:56,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:56,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-27 23:59:56,805 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=bec4bce8b8bdfe22904d678de5936bd8, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:56,806 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1698451196805"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451196805"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451196805"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451196805"}]},"ts":"1698451196805"} 2023-10-27 23:59:56,809 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=69, resume processing ppid=67 2023-10-27 23:59:56,809 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure bec4bce8b8bdfe22904d678de5936bd8, server=jenkins-hbase16.apache.org,43469,1698451096272 in 188 msec 2023-10-27 23:59:56,810 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=67, resume processing ppid=66 2023-10-27 23:59:56,810 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=bec4bce8b8bdfe22904d678de5936bd8, REOPEN/MOVE in 517 msec 2023-10-27 23:59:56,811 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=66, resume processing ppid=65 2023-10-27 23:59:56,811 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 521 msec 2023-10-27 23:59:56,812 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 944 msec 2023-10-27 23:59:56,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=65 2023-10-27 23:59:56,814 INFO [Listener at localhost/40295] compactions.TestMobCompactor(278): alter status finished 2023-10-27 23:59:56,815 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36413] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-10-27 23:59:56,816 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] procedure2.ProcedureExecutor(1028): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-10-27 23:59:56,816 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-10-27 23:59:56,817 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698451796817 2023-10-27 23:59:56,817 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-27 23:59:56,818 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-10-27 23:59:56,818 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-10-27 23:59:56,818 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-10-27 23:59:56,819 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] zookeeper.ReadOnlyZKClient(139): Connect 0x028abc53 to 127.0.0.1:57369 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-10-27 23:59:56,851 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1c4df59e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-27 23:59:56,851 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-10-27 23:59:56,851 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-10-27 23:59:56,851 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-10-27 23:59:56,851 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-10-27 23:59:56,851 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-10-27 23:59:56,851 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-10-27 23:59:56,851 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-10-27 23:59:56,852 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-10-27 23:59:56,852 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-10-27 23:59:56,852 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-27 23:59:56,852 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-10-27 23:59:56,853 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 37 msec 2023-10-27 23:59:56,904 INFO [Listener at localhost/40295] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=242 (was 249), OpenFileDescriptor=613 (was 626), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=126 (was 165), ProcessCount=164 (was 166), AvailableMemoryMB=6012 (was 6036) 2023-10-27 23:59:56,914 INFO [Listener at localhost/40295] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=242, OpenFileDescriptor=613, MaxFileDescriptor=60000, SystemLoadAverage=126, ProcessCount=164, AvailableMemoryMB=6012 2023-10-27 23:59:56,917 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.HMaster$4(2112): Client=jenkins//188.40.62.62 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-10-27 23:59:56,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure2.ProcedureExecutor(1028): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-10-27 23:59:56,919 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-10-27 23:59:56,919 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(700): Client=jenkins//188.40.62.62 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-10-27 23:59:56,920 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-10-27 23:59:56,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-10-27 23:59:56,921 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f 2023-10-27 23:59:56,921 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f empty. 2023-10-27 23:59:56,922 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f 2023-10-27 23:59:56,922 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-10-27 23:59:57,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-10-27 23:59:57,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-10-27 23:59:57,332 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-10-27 23:59:57,335 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => 7d63fcb50d6ff768a587de9ba4a6199f, NAME => 'testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp 2023-10-27 23:59:57,524 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-10-27 23:59:57,758 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:59:57,758 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1604): Closing 7d63fcb50d6ff768a587de9ba4a6199f, disabling compactions & flushes 2023-10-27 23:59:57,758 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f. 2023-10-27 23:59:57,758 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f. 2023-10-27 23:59:57,759 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f. after waiting 0 ms 2023-10-27 23:59:57,759 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f. 2023-10-27 23:59:57,759 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f. 2023-10-27 23:59:57,759 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1558): Region close journal for 7d63fcb50d6ff768a587de9ba4a6199f: 2023-10-27 23:59:57,764 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-10-27 23:59:57,767 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1698451197766"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451197766"}]},"ts":"1698451197766"} 2023-10-27 23:59:57,769 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-10-27 23:59:57,771 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-10-27 23:59:57,771 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451197771"}]},"ts":"1698451197771"} 2023-10-27 23:59:57,772 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-10-27 23:59:57,808 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=7d63fcb50d6ff768a587de9ba4a6199f, ASSIGN}] 2023-10-27 23:59:57,810 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=7d63fcb50d6ff768a587de9ba4a6199f, ASSIGN 2023-10-27 23:59:57,811 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=7d63fcb50d6ff768a587de9ba4a6199f, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,43469,1698451096272; forceNewPlan=false, retain=false 2023-10-27 23:59:57,962 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=7d63fcb50d6ff768a587de9ba4a6199f, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:57,962 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1698451197962"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451197962"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451197962"}]},"ts":"1698451197962"} 2023-10-27 23:59:57,965 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure 7d63fcb50d6ff768a587de9ba4a6199f, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-27 23:59:58,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-10-27 23:59:58,117 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:58,124 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f. 2023-10-27 23:59:58,125 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7d63fcb50d6ff768a587de9ba4a6199f, NAME => 'testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f.', STARTKEY => '', ENDKEY => ''} 2023-10-27 23:59:58,126 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-10-27 23:59:58,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-10-27 23:59:58,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction 7d63fcb50d6ff768a587de9ba4a6199f 2023-10-27 23:59:58,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-27 23:59:58,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for 7d63fcb50d6ff768a587de9ba4a6199f 2023-10-27 23:59:58,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for 7d63fcb50d6ff768a587de9ba4a6199f 2023-10-27 23:59:58,131 INFO [StoreOpener-7d63fcb50d6ff768a587de9ba4a6199f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region 7d63fcb50d6ff768a587de9ba4a6199f 2023-10-27 23:59:58,133 DEBUG [StoreOpener-7d63fcb50d6ff768a587de9ba4a6199f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1 2023-10-27 23:59:58,133 DEBUG [StoreOpener-7d63fcb50d6ff768a587de9ba4a6199f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1 2023-10-27 23:59:58,134 INFO [StoreOpener-7d63fcb50d6ff768a587de9ba4a6199f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7d63fcb50d6ff768a587de9ba4a6199f columnFamilyName f1 2023-10-27 23:59:58,136 INFO [StoreOpener-7d63fcb50d6ff768a587de9ba4a6199f-1] regionserver.HStore(310): Store=7d63fcb50d6ff768a587de9ba4a6199f/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-27 23:59:58,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f 2023-10-27 23:59:58,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f 2023-10-27 23:59:58,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for 7d63fcb50d6ff768a587de9ba4a6199f 2023-10-27 23:59:58,144 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-10-27 23:59:58,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened 7d63fcb50d6ff768a587de9ba4a6199f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75019016, jitterRate=0.11787045001983643}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-10-27 23:59:58,145 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for 7d63fcb50d6ff768a587de9ba4a6199f: 2023-10-27 23:59:58,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f., pid=73, masterSystemTime=1698451198117 2023-10-27 23:59:58,147 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f. 2023-10-27 23:59:58,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f. 2023-10-27 23:59:58,148 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=7d63fcb50d6ff768a587de9ba4a6199f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-27 23:59:58,148 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1698451198148"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451198148"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451198148"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451198148"}]},"ts":"1698451198148"} 2023-10-27 23:59:58,152 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=73, resume processing ppid=72 2023-10-27 23:59:58,152 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure 7d63fcb50d6ff768a587de9ba4a6199f, server=jenkins-hbase16.apache.org,43469,1698451096272 in 185 msec 2023-10-27 23:59:58,154 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=72, resume processing ppid=71 2023-10-27 23:59:58,154 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=7d63fcb50d6ff768a587de9ba4a6199f, ASSIGN in 344 msec 2023-10-27 23:59:58,155 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-10-27 23:59:58,155 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451198155"}]},"ts":"1698451198155"} 2023-10-27 23:59:58,156 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-10-27 23:59:58,181 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-10-27 23:59:58,183 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 1.2640 sec 2023-10-27 23:59:59,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-10-27 23:59:59,030 INFO [Listener at localhost/40295] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-10-27 23:59:59,031 DEBUG [Listener at localhost/40295] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-10-27 23:59:59,036 INFO [Listener at localhost/40295] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-10-27 23:59:59,041 INFO [Listener at localhost/40295] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-10-27 23:59:59,041 INFO [Listener at localhost/40295] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-10-27 23:59:59,041 INFO [Listener at localhost/40295] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-10-27 23:59:59,045 INFO [Listener at localhost/40295] regionserver.HRegion(2745): Flushing 7d63fcb50d6ff768a587de9ba4a6199f 1/1 column families, dataSize=37 B heapSize=368 B 2023-10-27 23:59:59,466 INFO [Listener at localhost/40295] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023102726d69db71aea46fc940a21e79e447865 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023102726d69db71aea46fc940a21e79e447865 2023-10-27 23:59:59,874 INFO [Listener at localhost/40295] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/.tmp/f1/e5799917780647b8b0d6ca204c4f2246 2023-10-27 23:59:59,885 DEBUG [Listener at localhost/40295] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/.tmp/f1/e5799917780647b8b0d6ca204c4f2246 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246 2023-10-27 23:59:59,889 INFO [Listener at localhost/40295] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246, entries=1, sequenceid=5, filesize=4.9 K 2023-10-27 23:59:59,890 INFO [Listener at localhost/40295] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 7d63fcb50d6ff768a587de9ba4a6199f in 845ms, sequenceid=5, compaction requested=false 2023-10-27 23:59:59,890 DEBUG [Listener at localhost/40295] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-10-27 23:59:59,891 DEBUG [Listener at localhost/40295] regionserver.HRegion(2446): Flush status journal for 7d63fcb50d6ff768a587de9ba4a6199f: 2023-10-27 23:59:59,893 INFO [Listener at localhost/40295] regionserver.HRegion(2745): Flushing 7d63fcb50d6ff768a587de9ba4a6199f 1/1 column families, dataSize=37 B heapSize=368 B 2023-10-28 00:00:00,060 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-10-28 00:00:00,305 INFO [Listener at localhost/40295] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231027d23af771fed0460da412f74664c214d3 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20231027d23af771fed0460da412f74664c214d3 2023-10-28 00:00:00,711 INFO [Listener at localhost/40295] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/.tmp/f1/53bb8a0a26724998901fe4cbf73fb9d8 2023-10-28 00:00:00,722 DEBUG [Listener at localhost/40295] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/.tmp/f1/53bb8a0a26724998901fe4cbf73fb9d8 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8 2023-10-28 00:00:00,728 INFO [Listener at localhost/40295] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8, entries=1, sequenceid=9, filesize=4.9 K 2023-10-28 00:00:00,729 INFO [Listener at localhost/40295] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 7d63fcb50d6ff768a587de9ba4a6199f in 836ms, sequenceid=9, compaction requested=false 2023-10-28 00:00:00,729 DEBUG [Listener at localhost/40295] regionserver.HRegion(2446): Flush status journal for 7d63fcb50d6ff768a587de9ba4a6199f: 2023-10-28 00:00:00,731 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36413] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-10-28 00:00:00,732 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] procedure2.ProcedureExecutor(1028): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-10-28 00:00:00,733 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-10-28 00:00:00,734 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698451800734 2023-10-28 00:00:00,734 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-28 00:00:00,735 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-10-28 00:00:00,735 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-10-28 00:00:00,735 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-10-28 00:00:00,736 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] zookeeper.ReadOnlyZKClient(139): Connect 0x72649274 to 127.0.0.1:57369 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-10-28 00:00:00,783 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:00,783 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:00,834 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:00,834 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:00,884 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:00,884 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:00,936 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:00,936 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:00,943 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1f678a3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-28 00:00:00,943 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20231027 for table='testGetAfterCompaction' and column='f1' 2023-10-28 00:00:00,987 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:00,987 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,038 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,038 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,089 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,089 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,140 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,140 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,191 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,192 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,242 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,243 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,294 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,294 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,345 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,345 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,396 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,396 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,448 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,448 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,499 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,499 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,549 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,549 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,600 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,600 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,651 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,651 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,702 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,702 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,753 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,753 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,779 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231027196403f475c3482e867df05a91c23fc1 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20231027196403f475c3482e867df05a91c23fc1 2023-10-28 00:00:01,780 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231027 table='testGetAfterCompaction' column='f1' 2023-10-28 00:00:01,794 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-10-28 00:00:01,795 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:50664, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-10-28 00:00:01,800 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-10-28 00:00:01,802 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:57898, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-10-28 00:00:01,804 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,804 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,810 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20231027/f1/fbc7464b1f3f4c95b054ed9b91f41032 first=Optional[r1] last=Optional[r2] 2023-10-28 00:00:01,811 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=2 for row with hfile group [{f1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20231027/f1/fbc7464b1f3f4c95b054ed9b91f41032}] 2023-10-28 00:00:01,814 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20231027/f1/fbc7464b1f3f4c95b054ed9b91f41032 for inclusion in 7d63fcb50d6ff768a587de9ba4a6199f/f1 2023-10-28 00:00:01,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-10-28 00:00:01,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(777): Region bounds: first= last= 2023-10-28 00:00:01,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for 7d63fcb50d6ff768a587de9ba4a6199f: 2023-10-28 00:00:01,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20231027/f1/fbc7464b1f3f4c95b054ed9b91f41032 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testGetAfterCompaction__9nu9a5qfam4j6526rcokbft4us600jcc3dn7f8qnc5v2ch1rd8bp1o5va9fte1ra/f1/fbc7464b1f3f4c95b054ed9b91f41032 2023-10-28 00:00:01,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testGetAfterCompaction__9nu9a5qfam4j6526rcokbft4us600jcc3dn7f8qnc5v2ch1rd8bp1o5va9fte1ra/f1/fbc7464b1f3f4c95b054ed9b91f41032 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/3f49238f6a1445fcaca606b90dbde60e_SeqId_12_ 2023-10-28 00:00:01,819 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testGetAfterCompaction__9nu9a5qfam4j6526rcokbft4us600jcc3dn7f8qnc5v2ch1rd8bp1o5va9fte1ra/f1/fbc7464b1f3f4c95b054ed9b91f41032 into 7d63fcb50d6ff768a587de9ba4a6199f/f1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/3f49238f6a1445fcaca606b90dbde60e_SeqId_12_ - updating store file list. 2023-10-28 00:00:01,822 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/3f49238f6a1445fcaca606b90dbde60e_SeqId_12_ into 7d63fcb50d6ff768a587de9ba4a6199f/f1 2023-10-28 00:00:01,822 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testGetAfterCompaction__9nu9a5qfam4j6526rcokbft4us600jcc3dn7f8qnc5v2ch1rd8bp1o5va9fte1ra/f1/fbc7464b1f3f4c95b054ed9b91f41032 into 7d63fcb50d6ff768a587de9ba4a6199f/f1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/3f49238f6a1445fcaca606b90dbde60e_SeqId_12_) 2023-10-28 00:00:01,822 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testGetAfterCompaction__9nu9a5qfam4j6526rcokbft4us600jcc3dn7f8qnc5v2ch1rd8bp1o5va9fte1ra/f1/fbc7464b1f3f4c95b054ed9b91f41032 2023-10-28 00:00:01,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(1912): 7d63fcb50d6ff768a587de9ba4a6199f/f1 is initiating minor compaction 2023-10-28 00:00:01,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@26effc18; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-10-28 00:00:01,823 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 7d63fcb50d6ff768a587de9ba4a6199f/f1 in testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f. 2023-10-28 00:00:01,823 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/3f49238f6a1445fcaca606b90dbde60e_SeqId_12_] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/.tmp, totalSize=10.0 K 2023-10-28 00:00:01,823 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 53bb8a0a26724998901fe4cbf73fb9d8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-10-28 00:00:01,823 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 3f49238f6a1445fcaca606b90dbde60e_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-10-28 00:00:01,825 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231027 table='testGetAfterCompaction' column='f1' 2023-10-28 00:00:01,825 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:01,827 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023102726d69db71aea46fc940a21e79e447865 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023102726d69db71aea46fc940a21e79e447865 2023-10-28 00:00:01,828 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20231027d23af771fed0460da412f74664c214d3 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20231027d23af771fed0460da412f74664c214d3 2023-10-28 00:00:01,828 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-10-28 00:00:01,828 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20231027' 2023-10-28 00:00:01,828 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20231027 for table='testGetAfterCompaction' and column='f1' 2023-10-28 00:00:01,828 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-10-28 00:00:01,828 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-28 00:00:01,829 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-10-28 00:00:01,830 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 1.0970 sec 2023-10-28 00:00:01,830 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] throttle.PressureAwareThroughputController(145): 7d63fcb50d6ff768a587de9ba4a6199f#f1#compaction#77 average throughput is 0.25 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 73.08 MB/second 2023-10-28 00:00:01,855 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/3f49238f6a1445fcaca606b90dbde60e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698451201776; access_time=1698451200965; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-10-28 00:00:01,855 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,855 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,859 DEBUG [Listener at localhost/40295] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-10-28 00:00:01,861 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:50666, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-10-28 00:00:01,862 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43469] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f. 2023-10-28 00:00:01,862 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43469] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 2 compacting, 0 eligible, 16 blocking 2023-10-28 00:00:01,862 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43469] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-10-28 00:00:01,862 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43469] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-10-28 00:00:01,862 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43469] regionserver.CompactSplit(415): Not compacting testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f. because compaction request was cancelled 2023-10-28 00:00:01,914 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/3f49238f6a1445fcaca606b90dbde60e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698451201776; access_time=1698451200965; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-10-28 00:00:01,914 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,914 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,965 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/3f49238f6a1445fcaca606b90dbde60e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698451201776; access_time=1698451200965; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-10-28 00:00:01,965 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:01,965 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,015 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/3f49238f6a1445fcaca606b90dbde60e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698451201776; access_time=1698451200965; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-10-28 00:00:02,016 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,016 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,066 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/3f49238f6a1445fcaca606b90dbde60e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698451201776; access_time=1698451200965; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-10-28 00:00:02,067 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,067 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,118 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/3f49238f6a1445fcaca606b90dbde60e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698451201776; access_time=1698451200965; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-10-28 00:00:02,118 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,118 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,168 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/3f49238f6a1445fcaca606b90dbde60e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698451201776; access_time=1698451200965; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-10-28 00:00:02,169 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,169 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,219 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/3f49238f6a1445fcaca606b90dbde60e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698451201776; access_time=1698451200965; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-10-28 00:00:02,219 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,219 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,270 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/3f49238f6a1445fcaca606b90dbde60e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698451201776; access_time=1698451200965; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-10-28 00:00:02,270 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,271 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,321 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/3f49238f6a1445fcaca606b90dbde60e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698451201776; access_time=1698451200965; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-10-28 00:00:02,321 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,321 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,372 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/3f49238f6a1445fcaca606b90dbde60e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698451201776; access_time=1698451200965; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-10-28 00:00:02,372 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,372 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,423 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/3f49238f6a1445fcaca606b90dbde60e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698451201776; access_time=1698451200965; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-10-28 00:00:02,423 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,423 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,474 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/3f49238f6a1445fcaca606b90dbde60e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698451201776; access_time=1698451200965; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-10-28 00:00:02,474 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,474 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,524 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/3f49238f6a1445fcaca606b90dbde60e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698451201776; access_time=1698451200965; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-10-28 00:00:02,525 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,525 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,575 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/3f49238f6a1445fcaca606b90dbde60e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698451201776; access_time=1698451200965; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-10-28 00:00:02,575 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,575 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,626 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/3f49238f6a1445fcaca606b90dbde60e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698451201776; access_time=1698451200965; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-10-28 00:00:02,627 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,627 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,644 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/.tmp/f1/617b428f780341f69ba141773f5cd1b4 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/617b428f780341f69ba141773f5cd1b4 2023-10-28 00:00:02,648 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in 7d63fcb50d6ff768a587de9ba4a6199f/f1 of 7d63fcb50d6ff768a587de9ba4a6199f into 617b428f780341f69ba141773f5cd1b4(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-10-28 00:00:02,648 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 7d63fcb50d6ff768a587de9ba4a6199f: 2023-10-28 00:00:02,648 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1698451201823; duration=0sec 2023-10-28 00:00:02,648 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-28 00:00:02,678 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/3f49238f6a1445fcaca606b90dbde60e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698451201776; access_time=1698451200965; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-10-28 00:00:02,678 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,678 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/617b428f780341f69ba141773f5cd1b4; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1698451202638; access_time=1698451201828; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,678 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,731 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/3f49238f6a1445fcaca606b90dbde60e_SeqId_12_] to archive 2023-10-28 00:00:02,731 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/3f49238f6a1445fcaca606b90dbde60e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698451201776; access_time=1698451200965; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-10-28 00:00:02,731 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451200711; access_time=1698451199895; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,731 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/617b428f780341f69ba141773f5cd1b4; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1698451202638; access_time=1698451201828; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,731 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,731 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:02,733 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/53bb8a0a26724998901fe4cbf73fb9d8 2023-10-28 00:00:02,734 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/3f49238f6a1445fcaca606b90dbde60e_SeqId_12_ to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/3f49238f6a1445fcaca606b90dbde60e_SeqId_12_ 2023-10-28 00:00:02,781 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/617b428f780341f69ba141773f5cd1b4; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1698451202638; access_time=1698451201828; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,782 INFO [Listener at localhost/40295] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698451199873; access_time=1698451199048; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-10-28 00:00:02,896 INFO [Listener at localhost/40295] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-10-28 00:00:02,899 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.HMaster$11(2418): Client=jenkins//188.40.62.62 disable testGetAfterCompaction 2023-10-28 00:00:02,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure2.ProcedureExecutor(1028): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-10-28 00:00:02,906 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451202906"}]},"ts":"1698451202906"} 2023-10-28 00:00:02,907 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-10-28 00:00:02,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=75 2023-10-28 00:00:02,950 INFO [PEWorker-2] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-10-28 00:00:02,951 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=7d63fcb50d6ff768a587de9ba4a6199f, UNASSIGN}] 2023-10-28 00:00:02,952 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=7d63fcb50d6ff768a587de9ba4a6199f, UNASSIGN 2023-10-28 00:00:02,953 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=7d63fcb50d6ff768a587de9ba4a6199f, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:02,953 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1698451202953"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451202953"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451202953"}]},"ts":"1698451202953"} 2023-10-28 00:00:02,953 WARN [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=36413] assignment.AssignmentManager(1312): Unable to acquire lock for regionNode state=CLOSING, location=jenkins-hbase16.apache.org,43469,1698451096272, table=testGetAfterCompaction, region=7d63fcb50d6ff768a587de9ba4a6199f. It is likely that another thread is currently holding the lock. To avoid deadlock, skip execution for now. 2023-10-28 00:00:02,954 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure 7d63fcb50d6ff768a587de9ba4a6199f, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-28 00:00:03,008 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=75 2023-10-28 00:00:03,105 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:03,106 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(111): Close 7d63fcb50d6ff768a587de9ba4a6199f 2023-10-28 00:00:03,106 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing 7d63fcb50d6ff768a587de9ba4a6199f, disabling compactions & flushes 2023-10-28 00:00:03,106 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f. 2023-10-28 00:00:03,106 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f. 2023-10-28 00:00:03,106 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f. after waiting 0 ms 2023-10-28 00:00:03,106 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f. 2023-10-28 00:00:03,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-10-28 00:00:03,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-10-28 00:00:03,113 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f. 2023-10-28 00:00:03,113 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for 7d63fcb50d6ff768a587de9ba4a6199f: 2023-10-28 00:00:03,115 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(149): Closed 7d63fcb50d6ff768a587de9ba4a6199f 2023-10-28 00:00:03,116 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=7d63fcb50d6ff768a587de9ba4a6199f, regionState=CLOSED 2023-10-28 00:00:03,116 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1698451203116"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451203116"}]},"ts":"1698451203116"} 2023-10-28 00:00:03,118 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=77, resume processing ppid=76 2023-10-28 00:00:03,119 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure 7d63fcb50d6ff768a587de9ba4a6199f, server=jenkins-hbase16.apache.org,43469,1698451096272 in 163 msec 2023-10-28 00:00:03,120 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=76, resume processing ppid=75 2023-10-28 00:00:03,120 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=7d63fcb50d6ff768a587de9ba4a6199f, UNASSIGN in 168 msec 2023-10-28 00:00:03,121 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451203120"}]},"ts":"1698451203120"} 2023-10-28 00:00:03,122 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-10-28 00:00:03,131 INFO [PEWorker-5] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-10-28 00:00:03,134 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 232 msec 2023-10-28 00:00:03,210 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=75 2023-10-28 00:00:03,210 INFO [Listener at localhost/40295] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-10-28 00:00:03,214 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.HMaster$5(2228): Client=jenkins//188.40.62.62 delete testGetAfterCompaction 2023-10-28 00:00:03,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure2.ProcedureExecutor(1028): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-10-28 00:00:03,220 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-10-28 00:00:03,222 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-10-28 00:00:03,223 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=78 2023-10-28 00:00:03,224 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f 2023-10-28 00:00:03,226 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1, FileablePath, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/recovered.edits] 2023-10-28 00:00:03,231 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/617b428f780341f69ba141773f5cd1b4 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/617b428f780341f69ba141773f5cd1b4 2023-10-28 00:00:03,232 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/f1/e5799917780647b8b0d6ca204c4f2246 2023-10-28 00:00:03,234 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/recovered.edits/15.seqid to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f/recovered.edits/15.seqid 2023-10-28 00:00:03,235 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testGetAfterCompaction/7d63fcb50d6ff768a587de9ba4a6199f 2023-10-28 00:00:03,235 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-10-28 00:00:03,235 DEBUG [PEWorker-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-10-28 00:00:03,236 DEBUG [PEWorker-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-10-28 00:00:03,239 DEBUG [PEWorker-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20231027196403f475c3482e867df05a91c23fc1 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20231027196403f475c3482e867df05a91c23fc1 2023-10-28 00:00:03,240 DEBUG [PEWorker-3] backup.HFileArchiver(596): Deleted hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-10-28 00:00:03,241 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-10-28 00:00:03,261 WARN [PEWorker-3] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-10-28 00:00:03,264 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-10-28 00:00:03,265 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-10-28 00:00:03,265 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-10-28 00:00:03,266 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1698451203265"}]},"ts":"9223372036854775807"} 2023-10-28 00:00:03,267 INFO [PEWorker-3] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-10-28 00:00:03,267 DEBUG [PEWorker-3] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 7d63fcb50d6ff768a587de9ba4a6199f, NAME => 'testGetAfterCompaction,,1698451196916.7d63fcb50d6ff768a587de9ba4a6199f.', STARTKEY => '', ENDKEY => ''}] 2023-10-28 00:00:03,267 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-10-28 00:00:03,268 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1698451203267"}]},"ts":"9223372036854775807"} 2023-10-28 00:00:03,270 INFO [PEWorker-3] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-10-28 00:00:03,282 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-10-28 00:00:03,283 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 67 msec 2023-10-28 00:00:03,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=78 2023-10-28 00:00:03,324 INFO [Listener at localhost/40295] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-10-28 00:00:03,335 INFO [Listener at localhost/40295] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=245 (was 242) - Thread LEAK? -, OpenFileDescriptor=606 (was 613), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=115 (was 126), ProcessCount=166 (was 164) - ProcessCount LEAK? -, AvailableMemoryMB=5970 (was 6012) 2023-10-28 00:00:03,344 INFO [Listener at localhost/40295] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=245, OpenFileDescriptor=606, MaxFileDescriptor=60000, SystemLoadAverage=115, ProcessCount=166, AvailableMemoryMB=5969 2023-10-28 00:00:03,346 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.HMaster$4(2112): Client=jenkins//188.40.62.62 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-10-28 00:00:03,347 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure2.ProcedureExecutor(1028): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-10-28 00:00:03,348 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-10-28 00:00:03,349 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(700): Client=jenkins//188.40.62.62 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-10-28 00:00:03,349 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-10-28 00:00:03,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-10-28 00:00:03,351 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea 2023-10-28 00:00:03,351 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96 2023-10-28 00:00:03,351 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf 2023-10-28 00:00:03,354 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea empty. 2023-10-28 00:00:03,354 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf empty. 2023-10-28 00:00:03,354 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96 empty. 2023-10-28 00:00:03,354 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea 2023-10-28 00:00:03,354 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf 2023-10-28 00:00:03,355 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96 2023-10-28 00:00:03,355 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-10-28 00:00:03,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-10-28 00:00:03,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-10-28 00:00:03,763 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-10-28 00:00:03,764 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7675): creating {ENCODED => 795595fcd155993336d014aba73423ea, NAME => 'testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp 2023-10-28 00:00:03,764 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7675): creating {ENCODED => 4904326d2e05286728bb6d35b919e5bf, NAME => 'testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp 2023-10-28 00:00:03,765 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7675): creating {ENCODED => 6d789d75232206c2589efabacffe9d96, NAME => 'testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp 2023-10-28 00:00:03,952 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-10-28 00:00:04,171 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-28 00:00:04,172 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-28 00:00:04,171 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-28 00:00:04,172 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1604): Closing 795595fcd155993336d014aba73423ea, disabling compactions & flushes 2023-10-28 00:00:04,172 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1604): Closing 6d789d75232206c2589efabacffe9d96, disabling compactions & flushes 2023-10-28 00:00:04,172 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:04,172 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:04,172 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. after waiting 0 ms 2023-10-28 00:00:04,172 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:04,172 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:04,172 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1558): Region close journal for 795595fcd155993336d014aba73423ea: 2023-10-28 00:00:04,172 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1604): Closing 4904326d2e05286728bb6d35b919e5bf, disabling compactions & flushes 2023-10-28 00:00:04,172 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:04,172 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:04,172 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. after waiting 0 ms 2023-10-28 00:00:04,172 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:04,172 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:04,172 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:04,172 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:04,173 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. after waiting 1 ms 2023-10-28 00:00:04,173 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1558): Region close journal for 4904326d2e05286728bb6d35b919e5bf: 2023-10-28 00:00:04,173 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:04,173 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:04,173 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1558): Region close journal for 6d789d75232206c2589efabacffe9d96: 2023-10-28 00:00:04,174 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-10-28 00:00:04,175 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698451204175"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451204175"}]},"ts":"1698451204175"} 2023-10-28 00:00:04,175 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698451204175"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451204175"}]},"ts":"1698451204175"} 2023-10-28 00:00:04,175 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1698451204175"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451204175"}]},"ts":"1698451204175"} 2023-10-28 00:00:04,177 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-10-28 00:00:04,177 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-10-28 00:00:04,178 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451204177"}]},"ts":"1698451204177"} 2023-10-28 00:00:04,178 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-10-28 00:00:04,218 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=795595fcd155993336d014aba73423ea, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6d789d75232206c2589efabacffe9d96, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=4904326d2e05286728bb6d35b919e5bf, ASSIGN}] 2023-10-28 00:00:04,219 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6d789d75232206c2589efabacffe9d96, ASSIGN 2023-10-28 00:00:04,219 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=795595fcd155993336d014aba73423ea, ASSIGN 2023-10-28 00:00:04,219 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=4904326d2e05286728bb6d35b919e5bf, ASSIGN 2023-10-28 00:00:04,220 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6d789d75232206c2589efabacffe9d96, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,43469,1698451096272; forceNewPlan=false, retain=false 2023-10-28 00:00:04,220 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=795595fcd155993336d014aba73423ea, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,43469,1698451096272; forceNewPlan=false, retain=false 2023-10-28 00:00:04,220 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=4904326d2e05286728bb6d35b919e5bf, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,43469,1698451096272; forceNewPlan=false, retain=false 2023-10-28 00:00:04,372 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=4904326d2e05286728bb6d35b919e5bf, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:04,372 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=6d789d75232206c2589efabacffe9d96, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:04,372 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698451204372"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451204372"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451204372"}]},"ts":"1698451204372"} 2023-10-28 00:00:04,372 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=795595fcd155993336d014aba73423ea, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:04,372 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1698451204372"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451204372"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451204372"}]},"ts":"1698451204372"} 2023-10-28 00:00:04,372 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698451204372"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451204372"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451204372"}]},"ts":"1698451204372"} 2023-10-28 00:00:04,374 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=83, ppid=82, state=RUNNABLE; OpenRegionProcedure 4904326d2e05286728bb6d35b919e5bf, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-28 00:00:04,374 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=84, ppid=81, state=RUNNABLE; OpenRegionProcedure 6d789d75232206c2589efabacffe9d96, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-28 00:00:04,376 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=85, ppid=80, state=RUNNABLE; OpenRegionProcedure 795595fcd155993336d014aba73423ea, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-28 00:00:04,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-10-28 00:00:04,527 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:04,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:04,530 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6d789d75232206c2589efabacffe9d96, NAME => 'testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96.', STARTKEY => '1', ENDKEY => '2'} 2023-10-28 00:00:04,530 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 6d789d75232206c2589efabacffe9d96 2023-10-28 00:00:04,530 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-28 00:00:04,530 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for 6d789d75232206c2589efabacffe9d96 2023-10-28 00:00:04,530 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for 6d789d75232206c2589efabacffe9d96 2023-10-28 00:00:04,531 INFO [StoreOpener-6d789d75232206c2589efabacffe9d96-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 6d789d75232206c2589efabacffe9d96 2023-10-28 00:00:04,533 DEBUG [StoreOpener-6d789d75232206c2589efabacffe9d96-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1 2023-10-28 00:00:04,533 DEBUG [StoreOpener-6d789d75232206c2589efabacffe9d96-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1 2023-10-28 00:00:04,533 INFO [StoreOpener-6d789d75232206c2589efabacffe9d96-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6d789d75232206c2589efabacffe9d96 columnFamilyName family1 2023-10-28 00:00:04,535 INFO [StoreOpener-6d789d75232206c2589efabacffe9d96-1] regionserver.HStore(310): Store=6d789d75232206c2589efabacffe9d96/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-28 00:00:04,535 INFO [StoreOpener-6d789d75232206c2589efabacffe9d96-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 6d789d75232206c2589efabacffe9d96 2023-10-28 00:00:04,536 DEBUG [StoreOpener-6d789d75232206c2589efabacffe9d96-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2 2023-10-28 00:00:04,536 DEBUG [StoreOpener-6d789d75232206c2589efabacffe9d96-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2 2023-10-28 00:00:04,536 INFO [StoreOpener-6d789d75232206c2589efabacffe9d96-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6d789d75232206c2589efabacffe9d96 columnFamilyName family2 2023-10-28 00:00:04,537 INFO [StoreOpener-6d789d75232206c2589efabacffe9d96-1] regionserver.HStore(310): Store=6d789d75232206c2589efabacffe9d96/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-28 00:00:04,538 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96 2023-10-28 00:00:04,539 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96 2023-10-28 00:00:04,540 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-10-28 00:00:04,541 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for 6d789d75232206c2589efabacffe9d96 2023-10-28 00:00:04,543 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-10-28 00:00:04,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened 6d789d75232206c2589efabacffe9d96; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74879179, jitterRate=0.11578671634197235}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-10-28 00:00:04,543 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for 6d789d75232206c2589efabacffe9d96: 2023-10-28 00:00:04,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96., pid=84, masterSystemTime=1698451204527 2023-10-28 00:00:04,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:04,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:04,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:04,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 795595fcd155993336d014aba73423ea, NAME => 'testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea.', STARTKEY => '', ENDKEY => '1'} 2023-10-28 00:00:04,545 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=6d789d75232206c2589efabacffe9d96, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:04,546 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1698451204545"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451204545"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451204545"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451204545"}]},"ts":"1698451204545"} 2023-10-28 00:00:04,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 795595fcd155993336d014aba73423ea 2023-10-28 00:00:04,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-28 00:00:04,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for 795595fcd155993336d014aba73423ea 2023-10-28 00:00:04,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for 795595fcd155993336d014aba73423ea 2023-10-28 00:00:04,548 INFO [StoreOpener-795595fcd155993336d014aba73423ea-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 795595fcd155993336d014aba73423ea 2023-10-28 00:00:04,548 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=84, resume processing ppid=81 2023-10-28 00:00:04,548 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=84, ppid=81, state=SUCCESS; OpenRegionProcedure 6d789d75232206c2589efabacffe9d96, server=jenkins-hbase16.apache.org,43469,1698451096272 in 173 msec 2023-10-28 00:00:04,549 DEBUG [StoreOpener-795595fcd155993336d014aba73423ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1 2023-10-28 00:00:04,549 DEBUG [StoreOpener-795595fcd155993336d014aba73423ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1 2023-10-28 00:00:04,549 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6d789d75232206c2589efabacffe9d96, ASSIGN in 332 msec 2023-10-28 00:00:04,549 INFO [StoreOpener-795595fcd155993336d014aba73423ea-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 795595fcd155993336d014aba73423ea columnFamilyName family1 2023-10-28 00:00:04,550 INFO [StoreOpener-795595fcd155993336d014aba73423ea-1] regionserver.HStore(310): Store=795595fcd155993336d014aba73423ea/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-28 00:00:04,550 INFO [StoreOpener-795595fcd155993336d014aba73423ea-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 795595fcd155993336d014aba73423ea 2023-10-28 00:00:04,551 DEBUG [StoreOpener-795595fcd155993336d014aba73423ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2 2023-10-28 00:00:04,551 DEBUG [StoreOpener-795595fcd155993336d014aba73423ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2 2023-10-28 00:00:04,551 INFO [StoreOpener-795595fcd155993336d014aba73423ea-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 795595fcd155993336d014aba73423ea columnFamilyName family2 2023-10-28 00:00:04,552 INFO [StoreOpener-795595fcd155993336d014aba73423ea-1] regionserver.HStore(310): Store=795595fcd155993336d014aba73423ea/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-28 00:00:04,553 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea 2023-10-28 00:00:04,553 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea 2023-10-28 00:00:04,554 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-10-28 00:00:04,555 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for 795595fcd155993336d014aba73423ea 2023-10-28 00:00:04,556 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-10-28 00:00:04,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened 795595fcd155993336d014aba73423ea; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67066577, jitterRate=-6.301254034042358E-4}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-10-28 00:00:04,556 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for 795595fcd155993336d014aba73423ea: 2023-10-28 00:00:04,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea., pid=85, masterSystemTime=1698451204527 2023-10-28 00:00:04,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:04,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:04,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:04,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4904326d2e05286728bb6d35b919e5bf, NAME => 'testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf.', STARTKEY => '2', ENDKEY => ''} 2023-10-28 00:00:04,558 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=795595fcd155993336d014aba73423ea, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:04,559 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 4904326d2e05286728bb6d35b919e5bf 2023-10-28 00:00:04,559 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698451204558"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451204558"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451204558"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451204558"}]},"ts":"1698451204558"} 2023-10-28 00:00:04,559 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-28 00:00:04,559 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for 4904326d2e05286728bb6d35b919e5bf 2023-10-28 00:00:04,559 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for 4904326d2e05286728bb6d35b919e5bf 2023-10-28 00:00:04,560 INFO [StoreOpener-4904326d2e05286728bb6d35b919e5bf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 4904326d2e05286728bb6d35b919e5bf 2023-10-28 00:00:04,561 DEBUG [StoreOpener-4904326d2e05286728bb6d35b919e5bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1 2023-10-28 00:00:04,561 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=85, resume processing ppid=80 2023-10-28 00:00:04,562 DEBUG [StoreOpener-4904326d2e05286728bb6d35b919e5bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1 2023-10-28 00:00:04,562 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=85, ppid=80, state=SUCCESS; OpenRegionProcedure 795595fcd155993336d014aba73423ea, server=jenkins-hbase16.apache.org,43469,1698451096272 in 185 msec 2023-10-28 00:00:04,562 INFO [StoreOpener-4904326d2e05286728bb6d35b919e5bf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4904326d2e05286728bb6d35b919e5bf columnFamilyName family1 2023-10-28 00:00:04,562 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=795595fcd155993336d014aba73423ea, ASSIGN in 346 msec 2023-10-28 00:00:04,563 INFO [StoreOpener-4904326d2e05286728bb6d35b919e5bf-1] regionserver.HStore(310): Store=4904326d2e05286728bb6d35b919e5bf/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-28 00:00:04,563 INFO [StoreOpener-4904326d2e05286728bb6d35b919e5bf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 4904326d2e05286728bb6d35b919e5bf 2023-10-28 00:00:04,564 DEBUG [StoreOpener-4904326d2e05286728bb6d35b919e5bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2 2023-10-28 00:00:04,564 DEBUG [StoreOpener-4904326d2e05286728bb6d35b919e5bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2 2023-10-28 00:00:04,565 INFO [StoreOpener-4904326d2e05286728bb6d35b919e5bf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4904326d2e05286728bb6d35b919e5bf columnFamilyName family2 2023-10-28 00:00:04,565 INFO [StoreOpener-4904326d2e05286728bb6d35b919e5bf-1] regionserver.HStore(310): Store=4904326d2e05286728bb6d35b919e5bf/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-28 00:00:04,566 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf 2023-10-28 00:00:04,567 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf 2023-10-28 00:00:04,568 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-10-28 00:00:04,569 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for 4904326d2e05286728bb6d35b919e5bf 2023-10-28 00:00:04,570 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-10-28 00:00:04,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened 4904326d2e05286728bb6d35b919e5bf; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63443284, jitterRate=-0.05462139844894409}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-10-28 00:00:04,571 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for 4904326d2e05286728bb6d35b919e5bf: 2023-10-28 00:00:04,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf., pid=83, masterSystemTime=1698451204527 2023-10-28 00:00:04,572 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:04,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:04,573 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=4904326d2e05286728bb6d35b919e5bf, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:04,573 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698451204573"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451204573"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451204573"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451204573"}]},"ts":"1698451204573"} 2023-10-28 00:00:04,575 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=83, resume processing ppid=82 2023-10-28 00:00:04,575 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=83, ppid=82, state=SUCCESS; OpenRegionProcedure 4904326d2e05286728bb6d35b919e5bf, server=jenkins-hbase16.apache.org,43469,1698451096272 in 201 msec 2023-10-28 00:00:04,576 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=82, resume processing ppid=79 2023-10-28 00:00:04,576 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=4904326d2e05286728bb6d35b919e5bf, ASSIGN in 357 msec 2023-10-28 00:00:04,576 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-10-28 00:00:04,577 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451204576"}]},"ts":"1698451204576"} 2023-10-28 00:00:04,577 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-10-28 00:00:04,609 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-10-28 00:00:04,610 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 1.2630 sec 2023-10-28 00:00:05,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-10-28 00:00:05,455 INFO [Listener at localhost/40295] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-10-28 00:00:05,462 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:00:05,462 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:00:05,463 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:00:05,465 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-28 00:00:05,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-10-28 00:00:05,471 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-10-28 00:00:05,472 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:05,472 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-10-28 00:00:05,472 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-10-28 00:00:05,472 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:05,472 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-10-28 00:00:05,500 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:05,500 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:05,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:05,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:05,500 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:05,500 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-28 00:00:05,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-10-28 00:00:05,501 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:05,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-28 00:00:05,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-10-28 00:00:05,501 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-10-28 00:00:05,501 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-10-28 00:00:05,501 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-10-28 00:00:05,501 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:05,501 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-10-28 00:00:05,502 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:05,502 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. started... 2023-10-28 00:00:05,502 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-10-28 00:00:05,502 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:05,502 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:05,502 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. started... 2023-10-28 00:00:05,502 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4904326d2e05286728bb6d35b919e5bf 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-10-28 00:00:05,502 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-28 00:00:05,503 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 795595fcd155993336d014aba73423ea 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-10-28 00:00:05,502 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. started... 2023-10-28 00:00:05,504 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 6d789d75232206c2589efabacffe9d96 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-10-28 00:00:05,515 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023102872405d973040401f805bd31e54b32525 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023102872405d973040401f805bd31e54b32525 2023-10-28 00:00:05,916 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231028b6ea31eec0664b918f7ee732a522247b to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231028b6ea31eec0664b918f7ee732a522247b 2023-10-28 00:00:05,916 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202310289d878da8c1b44dcc97673584130fa660 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202310289d878da8c1b44dcc97673584130fa660 2023-10-28 00:00:05,919 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/.tmp/family1/dcf9afa91e314f178c59ced5a25fa944 2023-10-28 00:00:06,088 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-10-28 00:00:06,329 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/.tmp/family1/9e88a17ec01944c9a13e6ecf7608bf19 2023-10-28 00:00:06,329 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/.tmp/family1/7236d93111f746b6b44c723fc9e64af8 2023-10-28 00:00:06,337 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231028a5ae3e73702f43c2bae66963f508b295 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231028a5ae3e73702f43c2bae66963f508b295 2023-10-28 00:00:06,745 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/.tmp/family2/2bfcfdf09d734e5dac83a9d255bd7680 2023-10-28 00:00:06,751 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023102881dc0532d5044ef7b13d871661b0cdfd to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023102881dc0532d5044ef7b13d871661b0cdfd 2023-10-28 00:00:06,751 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023102898ec894991c44e9a840d313061645c04 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023102898ec894991c44e9a840d313061645c04 2023-10-28 00:00:06,752 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/.tmp/family1/dcf9afa91e314f178c59ced5a25fa944 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/dcf9afa91e314f178c59ced5a25fa944 2023-10-28 00:00:06,758 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/dcf9afa91e314f178c59ced5a25fa944, entries=4, sequenceid=5, filesize=5.4 K 2023-10-28 00:00:06,759 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/.tmp/family2/2bfcfdf09d734e5dac83a9d255bd7680 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/2bfcfdf09d734e5dac83a9d255bd7680 2023-10-28 00:00:06,763 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/2bfcfdf09d734e5dac83a9d255bd7680, entries=2, sequenceid=5, filesize=5.1 K 2023-10-28 00:00:06,764 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 795595fcd155993336d014aba73423ea in 1261ms, sequenceid=5, compaction requested=false 2023-10-28 00:00:06,767 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 795595fcd155993336d014aba73423ea: 2023-10-28 00:00:06,767 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:06,768 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-10-28 00:00:07,160 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/.tmp/family2/f854e3377f064dfc805f6e6239b2242e 2023-10-28 00:00:07,170 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/.tmp/family2/974f6211ca1740f2a34fe077aa91bdf2 2023-10-28 00:00:07,173 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/.tmp/family1/7236d93111f746b6b44c723fc9e64af8 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/7236d93111f746b6b44c723fc9e64af8 2023-10-28 00:00:07,177 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/.tmp/family1/9e88a17ec01944c9a13e6ecf7608bf19 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/9e88a17ec01944c9a13e6ecf7608bf19 2023-10-28 00:00:07,179 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/7236d93111f746b6b44c723fc9e64af8, entries=4, sequenceid=5, filesize=5.4 K 2023-10-28 00:00:07,180 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/.tmp/family2/f854e3377f064dfc805f6e6239b2242e as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/f854e3377f064dfc805f6e6239b2242e 2023-10-28 00:00:07,182 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/9e88a17ec01944c9a13e6ecf7608bf19, entries=4, sequenceid=5, filesize=5.4 K 2023-10-28 00:00:07,183 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/.tmp/family2/974f6211ca1740f2a34fe077aa91bdf2 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/974f6211ca1740f2a34fe077aa91bdf2 2023-10-28 00:00:07,184 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/f854e3377f064dfc805f6e6239b2242e, entries=2, sequenceid=5, filesize=5.1 K 2023-10-28 00:00:07,185 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 6d789d75232206c2589efabacffe9d96 in 1681ms, sequenceid=5, compaction requested=false 2023-10-28 00:00:07,185 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 6d789d75232206c2589efabacffe9d96: 2023-10-28 00:00:07,185 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:07,185 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-10-28 00:00:07,186 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/974f6211ca1740f2a34fe077aa91bdf2, entries=2, sequenceid=5, filesize=5.1 K 2023-10-28 00:00:07,187 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 4904326d2e05286728bb6d35b919e5bf in 1685ms, sequenceid=5, compaction requested=false 2023-10-28 00:00:07,187 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4904326d2e05286728bb6d35b919e5bf: 2023-10-28 00:00:07,187 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:07,187 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-10-28 00:00:07,187 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-10-28 00:00:07,187 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:07,187 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-10-28 00:00:07,187 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-10-28 00:00:07,200 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:07,200 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:07,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:07,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:07,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:07,200 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:07,200 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-28 00:00:07,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:07,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:07,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:07,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:07,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:07,202 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-10-28 00:00:07,202 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4aeae73b[Count = 0] remaining members to acquire global barrier 2023-10-28 00:00:07,202 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-10-28 00:00:07,202 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:07,216 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:07,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:07,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:07,217 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-10-28 00:00:07,217 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-10-28 00:00:07,217 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-28 00:00:07,217 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:07,217 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-28 00:00:07,258 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-10-28 00:00:07,258 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:07,258 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:07,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:07,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:07,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:07,259 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-10-28 00:00:07,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:07,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:07,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:07,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:07,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:07,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:07,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:07,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-28 00:00:07,268 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-10-28 00:00:07,268 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-10-28 00:00:07,269 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-28 00:00:07,269 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-28 00:00:07,269 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-10-28 00:00:07,269 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-28 00:00:07,300 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:07,300 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:07,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:07,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:07,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:07,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:07,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:07,300 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:07,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:07,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:07,302 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:07,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:07,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:07,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:07,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:07,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:07,305 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:07,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:07,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:07,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:07,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:07,325 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:07,325 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:07,325 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-10-28 00:00:07,325 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-10-28 00:00:07,325 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-10-28 00:00:07,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:07,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:07,325 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:07,325 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:07,325 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-10-28 00:00:07,326 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-28 00:00:07,325 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:07,326 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:07,326 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:07,326 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:07,326 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-10-28 00:00:07,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:07,327 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-28 00:00:07,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:07,427 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-28 00:00:07,429 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-28 00:00:07,430 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:00:07,430 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:00:07,431 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:00:07,434 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-28 00:00:07,437 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-10-28 00:00:07,437 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-10-28 00:00:07,437 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-10-28 00:00:07,437 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:07,437 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-10-28 00:00:07,437 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-10-28 00:00:07,437 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:07,437 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-10-28 00:00:07,458 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:07,458 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:07,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:07,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:07,459 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:07,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-10-28 00:00:07,459 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-28 00:00:07,459 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:07,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-28 00:00:07,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-10-28 00:00:07,459 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-10-28 00:00:07,460 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-10-28 00:00:07,460 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-10-28 00:00:07,460 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-10-28 00:00:07,460 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:07,460 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-10-28 00:00:07,460 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:07,460 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-10-28 00:00:07,460 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-28 00:00:07,460 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:07,460 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. started... 2023-10-28 00:00:07,460 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:07,460 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. started... 2023-10-28 00:00:07,460 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. started... 2023-10-28 00:00:07,460 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4904326d2e05286728bb6d35b919e5bf 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-10-28 00:00:07,460 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 6d789d75232206c2589efabacffe9d96 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-10-28 00:00:07,460 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 795595fcd155993336d014aba73423ea 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-10-28 00:00:07,884 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231028625ac864940d4814bc5b572d46243862 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231028625ac864940d4814bc5b572d46243862 2023-10-28 00:00:07,884 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231028f1ccb541978d4e818c1e1cd3c99edddd to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231028f1ccb541978d4e818c1e1cd3c99edddd 2023-10-28 00:00:07,884 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202310283b7c07e40bf24d94844e40d254645d1f to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202310283b7c07e40bf24d94844e40d254645d1f 2023-10-28 00:00:08,293 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/.tmp/family1/4765103779e64954835a94c7a00e92a0 2023-10-28 00:00:08,293 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/.tmp/family1/30493979777049ce8ebd4cf3bc0cd5d5 2023-10-28 00:00:08,294 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/.tmp/family1/d39ca93432034ad29297bc42a5b0ea49 2023-10-28 00:00:08,617 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-10-28 00:00:08,778 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-10-28 00:00:08,778 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x72649274 to 127.0.0.1:57369 2023-10-28 00:00:08,778 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-10-28 00:00:08,780 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x028abc53 to 127.0.0.1:57369 2023-10-28 00:00:08,780 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-10-28 00:00:08,780 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-10-28 00:00:08,780 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0c6d1a6f to 127.0.0.1:57369 2023-10-28 00:00:08,780 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-10-28 00:00:08,784 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-10-28 00:00:08,784 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x07006e84 to 127.0.0.1:57369 2023-10-28 00:00:08,784 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-10-28 00:00:08,788 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231028f817b6d57d2c40f69a6e2c74a2770771 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231028f817b6d57d2c40f69a6e2c74a2770771 2023-10-28 00:00:08,788 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202310282f8d01bde2ac4b5b9f8db3d7eb0d5a77 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202310282f8d01bde2ac4b5b9f8db3d7eb0d5a77 2023-10-28 00:00:08,788 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231028b039874f3c504d3dbaa25299e35b785a to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231028b039874f3c504d3dbaa25299e35b785a 2023-10-28 00:00:09,198 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/.tmp/family2/2b1ba95eb4554512b2aa7f4f3bf684a7 2023-10-28 00:00:09,198 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/.tmp/family2/69967cf4dac4418fb3e0be94fc433f04 2023-10-28 00:00:09,199 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/.tmp/family2/2083b1a0e27a4323afa5cc383672554b 2023-10-28 00:00:09,207 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/.tmp/family1/30493979777049ce8ebd4cf3bc0cd5d5 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/30493979777049ce8ebd4cf3bc0cd5d5 2023-10-28 00:00:09,207 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/.tmp/family1/4765103779e64954835a94c7a00e92a0 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/4765103779e64954835a94c7a00e92a0 2023-10-28 00:00:09,207 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/.tmp/family1/d39ca93432034ad29297bc42a5b0ea49 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/d39ca93432034ad29297bc42a5b0ea49 2023-10-28 00:00:09,211 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/4765103779e64954835a94c7a00e92a0, entries=4, sequenceid=9, filesize=5.4 K 2023-10-28 00:00:09,212 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/30493979777049ce8ebd4cf3bc0cd5d5, entries=4, sequenceid=9, filesize=5.4 K 2023-10-28 00:00:09,212 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/d39ca93432034ad29297bc42a5b0ea49, entries=4, sequenceid=9, filesize=5.4 K 2023-10-28 00:00:09,212 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/.tmp/family2/2b1ba95eb4554512b2aa7f4f3bf684a7 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/2b1ba95eb4554512b2aa7f4f3bf684a7 2023-10-28 00:00:09,213 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/.tmp/family2/2083b1a0e27a4323afa5cc383672554b as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/2083b1a0e27a4323afa5cc383672554b 2023-10-28 00:00:09,213 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/.tmp/family2/69967cf4dac4418fb3e0be94fc433f04 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/69967cf4dac4418fb3e0be94fc433f04 2023-10-28 00:00:09,218 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/2b1ba95eb4554512b2aa7f4f3bf684a7, entries=2, sequenceid=9, filesize=5.1 K 2023-10-28 00:00:09,218 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/2083b1a0e27a4323afa5cc383672554b, entries=2, sequenceid=9, filesize=5.1 K 2023-10-28 00:00:09,218 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/69967cf4dac4418fb3e0be94fc433f04, entries=2, sequenceid=9, filesize=5.1 K 2023-10-28 00:00:09,219 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 4904326d2e05286728bb6d35b919e5bf in 1759ms, sequenceid=9, compaction requested=false 2023-10-28 00:00:09,219 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 795595fcd155993336d014aba73423ea in 1759ms, sequenceid=9, compaction requested=false 2023-10-28 00:00:09,219 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4904326d2e05286728bb6d35b919e5bf: 2023-10-28 00:00:09,219 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 795595fcd155993336d014aba73423ea: 2023-10-28 00:00:09,219 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:09,219 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:09,219 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 6d789d75232206c2589efabacffe9d96 in 1759ms, sequenceid=9, compaction requested=false 2023-10-28 00:00:09,220 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-10-28 00:00:09,220 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 6d789d75232206c2589efabacffe9d96: 2023-10-28 00:00:09,220 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-10-28 00:00:09,220 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:09,220 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-10-28 00:00:09,220 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-10-28 00:00:09,220 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:09,220 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-10-28 00:00:09,220 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-10-28 00:00:09,250 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:09,250 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:09,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:09,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:09,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:09,250 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:09,250 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-28 00:00:09,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:09,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:09,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:09,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:09,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:09,254 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-10-28 00:00:09,254 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@434b5707[Count = 0] remaining members to acquire global barrier 2023-10-28 00:00:09,254 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-10-28 00:00:09,254 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:09,265 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:09,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:09,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:09,265 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-10-28 00:00:09,265 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-10-28 00:00:09,265 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-28 00:00:09,266 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:09,266 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-28 00:00:09,275 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-10-28 00:00:09,275 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:09,275 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:09,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:09,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:09,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:09,275 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-10-28 00:00:09,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:09,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:09,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:09,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:09,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:09,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:09,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:09,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-28 00:00:09,286 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-10-28 00:00:09,286 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-10-28 00:00:09,286 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-28 00:00:09,286 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-28 00:00:09,286 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-10-28 00:00:09,286 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-28 00:00:09,298 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:09,298 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:09,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:09,298 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:09,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:09,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:09,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:09,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:09,299 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:09,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:09,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:09,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:09,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:09,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:09,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:09,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:09,300 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:09,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:09,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:09,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:09,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:09,308 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:09,308 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:09,308 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-10-28 00:00:09,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:09,308 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-10-28 00:00:09,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:09,308 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-10-28 00:00:09,308 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-28 00:00:09,308 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:09,308 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-10-28 00:00:09,308 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:09,308 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:09,308 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:09,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:09,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:09,308 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:09,309 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:09,309 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-10-28 00:00:09,309 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-28 00:00:09,409 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-28 00:00:09,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-28 00:00:09,411 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:00:09,411 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:00:09,412 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:00:09,414 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-28 00:00:09,416 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-10-28 00:00:09,416 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-10-28 00:00:09,416 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-10-28 00:00:09,417 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:09,417 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-10-28 00:00:09,417 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-10-28 00:00:09,417 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:09,417 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-10-28 00:00:09,424 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:09,424 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:09,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:09,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:09,425 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:09,425 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-28 00:00:09,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-10-28 00:00:09,425 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:09,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-28 00:00:09,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-10-28 00:00:09,426 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-10-28 00:00:09,426 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-10-28 00:00:09,426 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-10-28 00:00:09,426 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-10-28 00:00:09,426 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:09,426 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-10-28 00:00:09,426 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:09,426 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-10-28 00:00:09,427 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-28 00:00:09,426 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. started... 2023-10-28 00:00:09,426 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:09,427 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:09,427 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. started... 2023-10-28 00:00:09,427 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4904326d2e05286728bb6d35b919e5bf 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-10-28 00:00:09,427 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 6d789d75232206c2589efabacffe9d96 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-10-28 00:00:09,427 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. started... 2023-10-28 00:00:09,427 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 795595fcd155993336d014aba73423ea 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-10-28 00:00:09,451 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202310289c1bbedd5cfa43539015c979c7f943db to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202310289c1bbedd5cfa43539015c979c7f943db 2023-10-28 00:00:09,859 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202310289cf51282aea94d7d81e7ee5e8cb74330 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202310289cf51282aea94d7d81e7ee5e8cb74330 2023-10-28 00:00:09,859 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202310285de2655de28242baa74d1aa200b8e278 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202310285de2655de28242baa74d1aa200b8e278 2023-10-28 00:00:09,861 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/.tmp/family1/9a188e707c0a4959b84ee66b9e2b7085 2023-10-28 00:00:10,267 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/.tmp/family1/2cb1c76d629141179060550b5919816b 2023-10-28 00:00:10,267 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/.tmp/family1/486a9ea7cf6e48928fb3a88650c0f7e6 2023-10-28 00:00:10,279 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231028a720d1ccf6fe4f3ebc304645dfd29979 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231028a720d1ccf6fe4f3ebc304645dfd29979 2023-10-28 00:00:10,689 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/.tmp/family2/01571f5665324edb985a32da4c4341a4 2023-10-28 00:00:10,694 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231028c5f21f48281a40bbb55088320038093a to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231028c5f21f48281a40bbb55088320038093a 2023-10-28 00:00:10,694 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231028ebb1400b1e1248c28157257d7b397f33 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231028ebb1400b1e1248c28157257d7b397f33 2023-10-28 00:00:10,694 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/.tmp/family1/9a188e707c0a4959b84ee66b9e2b7085 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/9a188e707c0a4959b84ee66b9e2b7085 2023-10-28 00:00:10,699 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/9a188e707c0a4959b84ee66b9e2b7085, entries=4, sequenceid=13, filesize=5.4 K 2023-10-28 00:00:10,700 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/.tmp/family2/01571f5665324edb985a32da4c4341a4 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/01571f5665324edb985a32da4c4341a4 2023-10-28 00:00:10,703 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/01571f5665324edb985a32da4c4341a4, entries=2, sequenceid=13, filesize=5.1 K 2023-10-28 00:00:10,704 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 4904326d2e05286728bb6d35b919e5bf in 1277ms, sequenceid=13, compaction requested=true 2023-10-28 00:00:10,704 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4904326d2e05286728bb6d35b919e5bf: 2023-10-28 00:00:10,704 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:10,704 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-10-28 00:00:11,100 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/.tmp/family2/2948c86034f5417187a0810c68ad2363 2023-10-28 00:00:11,100 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/.tmp/family2/4a1c8f8ec1b744a293ebf977fdd7f04d 2023-10-28 00:00:11,112 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/.tmp/family1/486a9ea7cf6e48928fb3a88650c0f7e6 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/486a9ea7cf6e48928fb3a88650c0f7e6 2023-10-28 00:00:11,112 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/.tmp/family1/2cb1c76d629141179060550b5919816b as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/2cb1c76d629141179060550b5919816b 2023-10-28 00:00:11,117 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/2cb1c76d629141179060550b5919816b, entries=4, sequenceid=13, filesize=5.4 K 2023-10-28 00:00:11,117 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/486a9ea7cf6e48928fb3a88650c0f7e6, entries=4, sequenceid=13, filesize=5.4 K 2023-10-28 00:00:11,117 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/.tmp/family2/4a1c8f8ec1b744a293ebf977fdd7f04d as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/4a1c8f8ec1b744a293ebf977fdd7f04d 2023-10-28 00:00:11,117 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/.tmp/family2/2948c86034f5417187a0810c68ad2363 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/2948c86034f5417187a0810c68ad2363 2023-10-28 00:00:11,124 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/2948c86034f5417187a0810c68ad2363, entries=2, sequenceid=13, filesize=5.1 K 2023-10-28 00:00:11,124 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/4a1c8f8ec1b744a293ebf977fdd7f04d, entries=2, sequenceid=13, filesize=5.1 K 2023-10-28 00:00:11,125 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 6d789d75232206c2589efabacffe9d96 in 1698ms, sequenceid=13, compaction requested=true 2023-10-28 00:00:11,125 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 795595fcd155993336d014aba73423ea in 1698ms, sequenceid=13, compaction requested=true 2023-10-28 00:00:11,125 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 6d789d75232206c2589efabacffe9d96: 2023-10-28 00:00:11,125 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 795595fcd155993336d014aba73423ea: 2023-10-28 00:00:11,125 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:11,125 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:11,125 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-10-28 00:00:11,125 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-10-28 00:00:11,125 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-10-28 00:00:11,125 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:11,125 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-10-28 00:00:11,125 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-10-28 00:00:11,177 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:11,177 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:11,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:11,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:11,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:11,178 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:11,178 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-28 00:00:11,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:11,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:11,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:11,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:11,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:11,180 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-10-28 00:00:11,180 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@380ddb94[Count = 0] remaining members to acquire global barrier 2023-10-28 00:00:11,180 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-10-28 00:00:11,180 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:11,191 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:11,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:11,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:11,192 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-10-28 00:00:11,192 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-10-28 00:00:11,192 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:11,192 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-28 00:00:11,192 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-28 00:00:11,200 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-10-28 00:00:11,200 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:11,200 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-10-28 00:00:11,200 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:11,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:11,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:11,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:11,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:11,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:11,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:11,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:11,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:11,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:11,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:11,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-28 00:00:11,207 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-10-28 00:00:11,207 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-10-28 00:00:11,207 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-28 00:00:11,207 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-28 00:00:11,207 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-10-28 00:00:11,207 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-28 00:00:11,216 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:11,217 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:11,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:11,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:11,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:11,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:11,217 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:11,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:11,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:11,218 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:11,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:11,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:11,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:11,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:11,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:11,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:11,221 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:11,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:11,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:11,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:11,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:11,233 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:11,233 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-10-28 00:00:11,233 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-10-28 00:00:11,233 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:11,233 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-10-28 00:00:11,233 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-10-28 00:00:11,233 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:11,233 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-28 00:00:11,233 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:11,233 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:11,233 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:11,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:11,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:11,233 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:11,234 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-10-28 00:00:11,234 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-28 00:00:11,234 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:11,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:11,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:11,334 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-28 00:00:11,336 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-28 00:00:11,339 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:00:11,340 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:00:11,340 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:00:11,345 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-28 00:00:11,348 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-10-28 00:00:11,348 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-10-28 00:00:11,348 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-10-28 00:00:11,348 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:11,348 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-10-28 00:00:11,348 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-10-28 00:00:11,349 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:11,349 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-10-28 00:00:11,358 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:11,358 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:11,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:11,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:11,358 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:11,358 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-28 00:00:11,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-10-28 00:00:11,359 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:11,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-28 00:00:11,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-10-28 00:00:11,359 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-10-28 00:00:11,359 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-10-28 00:00:11,360 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-10-28 00:00:11,360 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-10-28 00:00:11,360 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:11,360 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-10-28 00:00:11,360 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:11,360 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:11,360 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-10-28 00:00:11,360 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:11,360 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. started... 2023-10-28 00:00:11,360 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. started... 2023-10-28 00:00:11,360 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-28 00:00:11,360 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. started... 2023-10-28 00:00:11,360 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 795595fcd155993336d014aba73423ea 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-10-28 00:00:11,360 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 6d789d75232206c2589efabacffe9d96 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-10-28 00:00:11,360 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4904326d2e05286728bb6d35b919e5bf 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-10-28 00:00:11,786 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231028d89c12a20837409db30f53e7647e3333 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231028d89c12a20837409db30f53e7647e3333 2023-10-28 00:00:11,786 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023102895f8921a8d4d4d278d029ad418c53fc9 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023102895f8921a8d4d4d278d029ad418c53fc9 2023-10-28 00:00:11,786 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202310284c7a22bee8b540ad908824b1f36d2ef3 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202310284c7a22bee8b540ad908824b1f36d2ef3 2023-10-28 00:00:11,795 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/.tmp/family1/1f3a75cc77e74b86b83067b7864d8a22 2023-10-28 00:00:12,193 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/.tmp/family1/1ac14e9e9ba54d8f8eae2adbb5a965e9 2023-10-28 00:00:12,193 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/.tmp/family1/c61d558f355843b0aec0e0a9e2c97536 2023-10-28 00:00:12,210 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023102821446b44bbb94f49ab9f91f9fa5050d6 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023102821446b44bbb94f49ab9f91f9fa5050d6 2023-10-28 00:00:12,617 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/.tmp/family2/af19a53e844a4102a55235df865fc030 2023-10-28 00:00:12,623 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023102869445068db394b3ebed604128d1a21e2 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023102869445068db394b3ebed604128d1a21e2 2023-10-28 00:00:12,624 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231028ceece078db4740dc8c8b77e7268b7665 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231028ceece078db4740dc8c8b77e7268b7665 2023-10-28 00:00:12,625 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/.tmp/family1/1f3a75cc77e74b86b83067b7864d8a22 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/1f3a75cc77e74b86b83067b7864d8a22 2023-10-28 00:00:12,630 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/.tmp/family2/1fc20602194041dda4e593b5943c1c21 2023-10-28 00:00:12,631 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/1f3a75cc77e74b86b83067b7864d8a22, entries=4, sequenceid=17, filesize=5.4 K 2023-10-28 00:00:12,631 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/.tmp/family2/af19a53e844a4102a55235df865fc030 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/af19a53e844a4102a55235df865fc030 2023-10-28 00:00:12,634 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/.tmp/family1/1ac14e9e9ba54d8f8eae2adbb5a965e9 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/1ac14e9e9ba54d8f8eae2adbb5a965e9 2023-10-28 00:00:12,635 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/af19a53e844a4102a55235df865fc030, entries=2, sequenceid=17, filesize=5.1 K 2023-10-28 00:00:12,636 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 4904326d2e05286728bb6d35b919e5bf in 1276ms, sequenceid=17, compaction requested=true 2023-10-28 00:00:12,636 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4904326d2e05286728bb6d35b919e5bf: 2023-10-28 00:00:12,636 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:12,636 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-10-28 00:00:12,637 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/1ac14e9e9ba54d8f8eae2adbb5a965e9, entries=4, sequenceid=17, filesize=5.4 K 2023-10-28 00:00:12,638 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/.tmp/family2/1fc20602194041dda4e593b5943c1c21 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/1fc20602194041dda4e593b5943c1c21 2023-10-28 00:00:12,642 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/1fc20602194041dda4e593b5943c1c21, entries=2, sequenceid=17, filesize=5.1 K 2023-10-28 00:00:12,643 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 795595fcd155993336d014aba73423ea in 1283ms, sequenceid=17, compaction requested=true 2023-10-28 00:00:12,643 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 795595fcd155993336d014aba73423ea: 2023-10-28 00:00:12,643 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:12,643 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-10-28 00:00:13,032 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/.tmp/family2/2f39466607184138b07dbe55159c1cf1 2023-10-28 00:00:13,041 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/.tmp/family1/c61d558f355843b0aec0e0a9e2c97536 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/c61d558f355843b0aec0e0a9e2c97536 2023-10-28 00:00:13,045 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/c61d558f355843b0aec0e0a9e2c97536, entries=4, sequenceid=17, filesize=5.4 K 2023-10-28 00:00:13,046 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/.tmp/family2/2f39466607184138b07dbe55159c1cf1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/2f39466607184138b07dbe55159c1cf1 2023-10-28 00:00:13,050 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/2f39466607184138b07dbe55159c1cf1, entries=2, sequenceid=17, filesize=5.1 K 2023-10-28 00:00:13,051 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 6d789d75232206c2589efabacffe9d96 in 1691ms, sequenceid=17, compaction requested=true 2023-10-28 00:00:13,051 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 6d789d75232206c2589efabacffe9d96: 2023-10-28 00:00:13,051 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:13,051 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-10-28 00:00:13,051 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-10-28 00:00:13,051 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:13,051 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-10-28 00:00:13,051 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-10-28 00:00:13,091 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:13,091 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:13,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:13,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:13,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:13,092 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:13,092 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-28 00:00:13,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:13,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:13,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:13,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:13,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:13,094 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-10-28 00:00:13,094 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@638dd4ff[Count = 0] remaining members to acquire global barrier 2023-10-28 00:00:13,094 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-10-28 00:00:13,094 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:13,099 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:13,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:13,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:13,100 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-10-28 00:00:13,100 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-10-28 00:00:13,100 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-28 00:00:13,100 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:13,100 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-28 00:00:13,108 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:13,108 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-10-28 00:00:13,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:13,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:13,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:13,108 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:13,108 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-10-28 00:00:13,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:13,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:13,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:13,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:13,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:13,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:13,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:13,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-28 00:00:13,112 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-10-28 00:00:13,112 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-28 00:00:13,112 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-28 00:00:13,112 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-10-28 00:00:13,112 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-10-28 00:00:13,112 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-28 00:00:13,123 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:13,123 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:13,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:13,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:13,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:13,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:13,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:13,123 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:13,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:13,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:13,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:13,125 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:13,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:13,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:13,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:13,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:13,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:13,128 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:13,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:13,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:13,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:13,141 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:13,141 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:13,141 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:13,141 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-10-28 00:00:13,141 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-10-28 00:00:13,141 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-10-28 00:00:13,141 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-28 00:00:13,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:13,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:13,141 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:13,141 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-10-28 00:00:13,142 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:13,142 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:13,142 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:13,142 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:13,142 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-10-28 00:00:13,142 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-28 00:00:13,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:13,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:13,242 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-28 00:00:13,243 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-28 00:00:13,255 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1583): Client=jenkins//188.40.62.62 snapshot request for:{ ss=snaptb-1698451213243 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-10-28 00:00:13,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1698451213255 (current time:1698451213255). 2023-10-28 00:00:13,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-10-28 00:00:13,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] zookeeper.ReadOnlyZKClient(139): Connect 0x1f1ebf7e to 127.0.0.1:57369 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-10-28 00:00:13,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2f4ad739, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-28 00:00:13,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-10-28 00:00:13,276 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:58050, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-10-28 00:00:13,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1f1ebf7e to 127.0.0.1:57369 2023-10-28 00:00:13,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] ipc.AbstractRpcClient(494): Stopping rpc client 2023-10-28 00:00:13,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] snapshot.SnapshotManager(637): No existing snapshot, attempting snapshot... 2023-10-28 00:00:13,278 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] snapshot.SnapshotManager(684): Table enabled, starting distributed snapshots for { ss=snaptb-1698451213243 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-10-28 00:00:13,287 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] snapshot.SnapshotManager(519): Unable to retrieve ACL status for path: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.hbase-snapshot, current working dir path: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.hbase-snapshot/.tmp/snaptb-1698451213243 org.apache.hadoop.hdfs.protocol.AclException: The ACL operation has been rejected. Support for ACLs has been disabled by setting dfs.namenode.acls.enabled to false. at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.checkAclsConfigFlag(FSDirAclOp.java:197) at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.getAclStatus(FSDirAclOp.java:150) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:6744) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getAclStatus(NameNodeRpcServer.java:2065) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getAclStatus(ClientNamenodeProtocolServerSideTranslatorPB.java:1378) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.getAclStatus(DFSClient.java:2736) at org.apache.hadoop.hdfs.DistributedFileSystem$51.doCall(DistributedFileSystem.java:2294) at org.apache.hadoop.hdfs.DistributedFileSystem$51.doCall(DistributedFileSystem.java:2291) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getAclStatus(DistributedFileSystem.java:2291) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.updateWorkingDirAclsIfRequired(SnapshotManager.java:517) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.prepareToTakeSnapshot(SnapshotManager.java:490) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.snapshotEnabledTable(SnapshotManager.java:559) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.takeSnapshotInternal(SnapshotManager.java:687) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.takeSnapshot(SnapshotManager.java:623) at org.apache.hadoop.hbase.master.MasterRpcServices.snapshot(MasterRpcServices.java:1588) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.AclException): The ACL operation has been rejected. Support for ACLs has been disabled by setting dfs.namenode.acls.enabled to false. at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.checkAclsConfigFlag(FSDirAclOp.java:197) at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.getAclStatus(FSDirAclOp.java:150) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:6744) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getAclStatus(NameNodeRpcServer.java:2065) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getAclStatus(ClientNamenodeProtocolServerSideTranslatorPB.java:1378) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1540) at org.apache.hadoop.ipc.Client.call(Client.java:1486) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy29.getAclStatus(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getAclStatus(ClientNamenodeProtocolTranslatorPB.java:1385) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy30.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.getAclStatus(DFSClient.java:2734) ... 15 more 2023-10-28 00:00:13,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure2.ProcedureExecutor(1028): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-10-28 00:00:13,294 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-10-28 00:00:13,294 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1698451813294 2023-10-28 00:00:13,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] snapshot.SnapshotManager(689): Started snapshot: { ss=snaptb-1698451213243 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-10-28 00:00:13,295 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1698451213243 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-10-28 00:00:13,295 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-10-28 00:00:13,296 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] procedure2.ProcedureExecutor(1028): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-10-28 00:00:13,297 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 3 msec 2023-10-28 00:00:13,297 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-10-28 00:00:13,297 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1698451813297 2023-10-28 00:00:13,300 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1698451213243 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-10-28 00:00:13,300 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-10-28 00:00:13,400 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-10-28 00:00:13,403 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=snaptb-1698451213243 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-10-28 00:00:13,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] snapshot.SnapshotManager(406): Snapshoting '{ ss=snaptb-1698451213243 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-10-28 00:00:13,406 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-10-28 00:00:13,606 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-10-28 00:00:13,609 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=snaptb-1698451213243 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-10-28 00:00:13,609 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] snapshot.SnapshotManager(406): Snapshoting '{ ss=snaptb-1698451213243 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-10-28 00:00:13,610 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-10-28 00:00:13,709 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1698451213243 2023-10-28 00:00:13,710 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1698451213243' 2023-10-28 00:00:13,710 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-10-28 00:00:13,710 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1698451213243' starting 'acquire' 2023-10-28 00:00:13,710 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1698451213243', kicking off acquire phase on members. 2023-10-28 00:00:13,712 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1698451213243 2023-10-28 00:00:13,712 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1698451213243 2023-10-28 00:00:13,725 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-10-28 00:00:13,725 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1698451213243/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:13,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-10-28 00:00:13,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-10-28 00:00:13,725 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1698451213243/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:13,726 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-28 00:00:13,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1698451213243 2023-10-28 00:00:13,727 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1698451213243 2023-10-28 00:00:13,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-10-28 00:00:13,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1698451213243 2023-10-28 00:00:13,728 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1698451213243 from table testCompactionWithHFileLink type FLUSH 2023-10-28 00:00:13,731 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1698451213243 2023-10-28 00:00:13,732 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1698451213243' with timeout 300000ms 2023-10-28 00:00:13,732 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-10-28 00:00:13,732 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1698451213243' starting 'acquire' stage 2023-10-28 00:00:13,732 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1698451213243' locally acquired 2023-10-28 00:00:13,732 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (snaptb-1698451213243) in zk 2023-10-28 00:00:13,741 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1698451213243 2023-10-28 00:00:13,741 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1698451213243/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:13,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1698451213243/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:13,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:13,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-10-28 00:00:13,741 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1698451213243 2023-10-28 00:00:13,742 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1698451213243' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-28 00:00:13,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:13,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:13,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1698451213243 2023-10-28 00:00:13,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:13,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:13,743 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'snaptb-1698451213243' on coordinator 2023-10-28 00:00:13,743 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7ec3248a[Count = 0] remaining members to acquire global barrier 2023-10-28 00:00:13,743 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1698451213243' starting 'in-barrier' execution. 2023-10-28 00:00:13,744 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1698451213243 2023-10-28 00:00:13,749 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1698451213243 2023-10-28 00:00:13,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1698451213243 2023-10-28 00:00:13,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1698451213243 2023-10-28 00:00:13,750 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1698451213243' received 'reached' from coordinator. 2023-10-28 00:00:13,750 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1698451213243/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:13,750 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-28 00:00:13,751 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:13,751 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:13,751 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-10-28 00:00:13,751 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:13,751 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-10-28 00:00:13,752 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. started... 2023-10-28 00:00:13,752 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. started... 2023-10-28 00:00:13,752 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. started... 2023-10-28 00:00:13,752 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-2] regionserver.HRegion(2446): Flush status journal for 795595fcd155993336d014aba73423ea: 2023-10-28 00:00:13,752 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-0] regionserver.HRegion(2446): Flush status journal for 4904326d2e05286728bb6d35b919e5bf: 2023-10-28 00:00:13,752 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-1] regionserver.HRegion(2446): Flush status journal for 6d789d75232206c2589efabacffe9d96: 2023-10-28 00:00:13,753 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea.' region-info for snapshot=snaptb-1698451213243 2023-10-28 00:00:13,753 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96.' region-info for snapshot=snaptb-1698451213243 2023-10-28 00:00:13,753 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf.' region-info for snapshot=snaptb-1698451213243 2023-10-28 00:00:13,760 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-10-28 00:00:13,760 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-10-28 00:00:13,760 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-10-28 00:00:13,764 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/dcf9afa91e314f178c59ced5a25fa944, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/30493979777049ce8ebd4cf3bc0cd5d5, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/2cb1c76d629141179060550b5919816b, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/1ac14e9e9ba54d8f8eae2adbb5a965e9] hfiles 2023-10-28 00:00:13,764 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/9e88a17ec01944c9a13e6ecf7608bf19, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/4765103779e64954835a94c7a00e92a0, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/9a188e707c0a4959b84ee66b9e2b7085, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/1f3a75cc77e74b86b83067b7864d8a22] hfiles 2023-10-28 00:00:13,764 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/9e88a17ec01944c9a13e6ecf7608bf19 for snapshot=snaptb-1698451213243 2023-10-28 00:00:13,764 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/7236d93111f746b6b44c723fc9e64af8, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/d39ca93432034ad29297bc42a5b0ea49, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/486a9ea7cf6e48928fb3a88650c0f7e6, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/c61d558f355843b0aec0e0a9e2c97536] hfiles 2023-10-28 00:00:13,764 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/7236d93111f746b6b44c723fc9e64af8 for snapshot=snaptb-1698451213243 2023-10-28 00:00:13,764 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/dcf9afa91e314f178c59ced5a25fa944 for snapshot=snaptb-1698451213243 2023-10-28 00:00:13,768 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/4765103779e64954835a94c7a00e92a0 for snapshot=snaptb-1698451213243 2023-10-28 00:00:13,768 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/9a188e707c0a4959b84ee66b9e2b7085 for snapshot=snaptb-1698451213243 2023-10-28 00:00:13,768 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/1f3a75cc77e74b86b83067b7864d8a22 for snapshot=snaptb-1698451213243 2023-10-28 00:00:13,768 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/30493979777049ce8ebd4cf3bc0cd5d5 for snapshot=snaptb-1698451213243 2023-10-28 00:00:13,768 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/974f6211ca1740f2a34fe077aa91bdf2, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/2b1ba95eb4554512b2aa7f4f3bf684a7, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/01571f5665324edb985a32da4c4341a4, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/af19a53e844a4102a55235df865fc030] hfiles 2023-10-28 00:00:13,768 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/974f6211ca1740f2a34fe077aa91bdf2 for snapshot=snaptb-1698451213243 2023-10-28 00:00:13,768 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/2b1ba95eb4554512b2aa7f4f3bf684a7 for snapshot=snaptb-1698451213243 2023-10-28 00:00:13,768 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/01571f5665324edb985a32da4c4341a4 for snapshot=snaptb-1698451213243 2023-10-28 00:00:13,768 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/af19a53e844a4102a55235df865fc030 for snapshot=snaptb-1698451213243 2023-10-28 00:00:13,768 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/d39ca93432034ad29297bc42a5b0ea49 for snapshot=snaptb-1698451213243 2023-10-28 00:00:13,768 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/2cb1c76d629141179060550b5919816b for snapshot=snaptb-1698451213243 2023-10-28 00:00:13,768 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/486a9ea7cf6e48928fb3a88650c0f7e6 for snapshot=snaptb-1698451213243 2023-10-28 00:00:13,768 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/1ac14e9e9ba54d8f8eae2adbb5a965e9 for snapshot=snaptb-1698451213243 2023-10-28 00:00:13,768 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/c61d558f355843b0aec0e0a9e2c97536 for snapshot=snaptb-1698451213243 2023-10-28 00:00:13,768 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/2bfcfdf09d734e5dac83a9d255bd7680, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/2083b1a0e27a4323afa5cc383672554b, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/4a1c8f8ec1b744a293ebf977fdd7f04d, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/1fc20602194041dda4e593b5943c1c21] hfiles 2023-10-28 00:00:13,768 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/f854e3377f064dfc805f6e6239b2242e, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/69967cf4dac4418fb3e0be94fc433f04, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/2948c86034f5417187a0810c68ad2363, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/2f39466607184138b07dbe55159c1cf1] hfiles 2023-10-28 00:00:13,768 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/2bfcfdf09d734e5dac83a9d255bd7680 for snapshot=snaptb-1698451213243 2023-10-28 00:00:13,768 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/f854e3377f064dfc805f6e6239b2242e for snapshot=snaptb-1698451213243 2023-10-28 00:00:13,768 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/2083b1a0e27a4323afa5cc383672554b for snapshot=snaptb-1698451213243 2023-10-28 00:00:13,769 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/69967cf4dac4418fb3e0be94fc433f04 for snapshot=snaptb-1698451213243 2023-10-28 00:00:13,769 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/4a1c8f8ec1b744a293ebf977fdd7f04d for snapshot=snaptb-1698451213243 2023-10-28 00:00:13,769 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/2948c86034f5417187a0810c68ad2363 for snapshot=snaptb-1698451213243 2023-10-28 00:00:13,769 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/1fc20602194041dda4e593b5943c1c21 for snapshot=snaptb-1698451213243 2023-10-28 00:00:13,769 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/2f39466607184138b07dbe55159c1cf1 for snapshot=snaptb-1698451213243 2023-10-28 00:00:13,910 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-10-28 00:00:13,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=snaptb-1698451213243 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-10-28 00:00:13,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] snapshot.SnapshotManager(406): Snapshoting '{ ss=snaptb-1698451213243 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-10-28 00:00:13,912 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-10-28 00:00:14,176 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. completed. 2023-10-28 00:00:14,176 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:14,177 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-10-28 00:00:14,177 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. completed. 2023-10-28 00:00:14,177 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:14,177 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-10-28 00:00:14,183 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. completed. 2023-10-28 00:00:14,183 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:14,183 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-10-28 00:00:14,184 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-10-28 00:00:14,184 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:14,184 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1698451213243' locally completed 2023-10-28 00:00:14,184 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1698451213243' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-28 00:00:14,216 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1698451213243/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:14,217 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1698451213243' has notified controller of completion 2023-10-28 00:00:14,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1698451213243/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:14,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:14,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-10-28 00:00:14,217 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:14,217 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1698451213243' completed. 2023-10-28 00:00:14,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:14,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:14,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1698451213243 2023-10-28 00:00:14,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:14,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:14,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1698451213243 2023-10-28 00:00:14,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:14,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1698451213243' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-28 00:00:14,228 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'snaptb-1698451213243', counting down latch. Waiting for 0 more 2023-10-28 00:00:14,228 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1698451213243' execution completed 2023-10-28 00:00:14,228 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-28 00:00:14,228 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-28 00:00:14,228 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1698451213243 2023-10-28 00:00:14,228 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1698451213243including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-10-28 00:00:14,240 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1698451213243 2023-10-28 00:00:14,240 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1698451213243 2023-10-28 00:00:14,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1698451213243 2023-10-28 00:00:14,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1698451213243 2023-10-28 00:00:14,240 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-10-28 00:00:14,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1698451213243 2023-10-28 00:00:14,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:14,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-10-28 00:00:14,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-10-28 00:00:14,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-10-28 00:00:14,242 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1698451213243/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:14,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:14,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1698451213243 2023-10-28 00:00:14,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1698451213243 2023-10-28 00:00:14,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:14,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1698451213243 2023-10-28 00:00:14,245 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1698451213243/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:14,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:14,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:14,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1698451213243 2023-10-28 00:00:14,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:14,258 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-10-28 00:00:14,258 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1698451213243/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:14,258 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1698451213243 2023-10-28 00:00:14,258 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1698451213243 2023-10-28 00:00:14,258 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/online-snapshot/abort/snaptb-1698451213243 because node does not exist (not an error) 2023-10-28 00:00:14,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-10-28 00:00:14,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-10-28 00:00:14,258 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-10-28 00:00:14,258 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1698451213243/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:14,258 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1698451213243 2023-10-28 00:00:14,258 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1698451213243 2023-10-28 00:00:14,258 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:14,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-10-28 00:00:14,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-10-28 00:00:14,259 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-10-28 00:00:14,259 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1698451213243 2023-10-28 00:00:14,259 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1698451213243 2023-10-28 00:00:14,259 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1698451213243 2023-10-28 00:00:14,259 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-10-28 00:00:14,261 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202310285de2655de28242baa74d1aa200b8e278, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231028b6ea31eec0664b918f7ee732a522247b, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231028d89c12a20837409db30f53e7647e3333, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231028f1ccb541978d4e818c1e1cd3c99edddd, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202310283b7c07e40bf24d94844e40d254645d1f, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023102895f8921a8d4d4d278d029ad418c53fc9, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202310289c1bbedd5cfa43539015c979c7f943db, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202310289d878da8c1b44dcc97673584130fa660, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202310284c7a22bee8b540ad908824b1f36d2ef3, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231028625ac864940d4814bc5b572d46243862, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023102872405d973040401f805bd31e54b32525, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202310289cf51282aea94d7d81e7ee5e8cb74330] mob files 2023-10-28 00:00:14,261 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202310285de2655de28242baa74d1aa200b8e278 2023-10-28 00:00:14,261 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231028b6ea31eec0664b918f7ee732a522247b 2023-10-28 00:00:14,261 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231028d89c12a20837409db30f53e7647e3333 2023-10-28 00:00:14,261 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231028f1ccb541978d4e818c1e1cd3c99edddd 2023-10-28 00:00:14,261 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202310283b7c07e40bf24d94844e40d254645d1f 2023-10-28 00:00:14,261 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023102895f8921a8d4d4d278d029ad418c53fc9 2023-10-28 00:00:14,261 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202310289c1bbedd5cfa43539015c979c7f943db 2023-10-28 00:00:14,262 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202310289d878da8c1b44dcc97673584130fa660 2023-10-28 00:00:14,262 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202310284c7a22bee8b540ad908824b1f36d2ef3 2023-10-28 00:00:14,262 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231028625ac864940d4814bc5b572d46243862 2023-10-28 00:00:14,262 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023102872405d973040401f805bd31e54b32525 2023-10-28 00:00:14,262 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202310289cf51282aea94d7d81e7ee5e8cb74330 2023-10-28 00:00:14,263 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202310282f8d01bde2ac4b5b9f8db3d7eb0d5a77, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023102869445068db394b3ebed604128d1a21e2, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023102881dc0532d5044ef7b13d871661b0cdfd, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231028ebb1400b1e1248c28157257d7b397f33, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023102821446b44bbb94f49ab9f91f9fa5050d6, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023102898ec894991c44e9a840d313061645c04, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231028a720d1ccf6fe4f3ebc304645dfd29979, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231028f817b6d57d2c40f69a6e2c74a2770771, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231028a5ae3e73702f43c2bae66963f508b295, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231028b039874f3c504d3dbaa25299e35b785a, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231028c5f21f48281a40bbb55088320038093a, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231028ceece078db4740dc8c8b77e7268b7665] mob files 2023-10-28 00:00:14,263 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202310282f8d01bde2ac4b5b9f8db3d7eb0d5a77 2023-10-28 00:00:14,263 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023102869445068db394b3ebed604128d1a21e2 2023-10-28 00:00:14,263 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023102881dc0532d5044ef7b13d871661b0cdfd 2023-10-28 00:00:14,263 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231028ebb1400b1e1248c28157257d7b397f33 2023-10-28 00:00:14,264 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023102821446b44bbb94f49ab9f91f9fa5050d6 2023-10-28 00:00:14,264 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023102898ec894991c44e9a840d313061645c04 2023-10-28 00:00:14,264 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231028a720d1ccf6fe4f3ebc304645dfd29979 2023-10-28 00:00:14,264 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231028f817b6d57d2c40f69a6e2c74a2770771 2023-10-28 00:00:14,264 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231028a5ae3e73702f43c2bae66963f508b295 2023-10-28 00:00:14,264 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231028b039874f3c504d3dbaa25299e35b785a 2023-10-28 00:00:14,264 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231028c5f21f48281a40bbb55088320038093a 2023-10-28 00:00:14,264 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231028ceece078db4740dc8c8b77e7268b7665 2023-10-28 00:00:14,412 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-10-28 00:00:14,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=snaptb-1698451213243 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-10-28 00:00:14,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] snapshot.SnapshotManager(406): Snapshoting '{ ss=snaptb-1698451213243 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-10-28 00:00:14,415 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-10-28 00:00:14,674 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1698451213243 2023-10-28 00:00:14,680 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.hbase-snapshot/.tmp/snaptb-1698451213243 2023-10-28 00:00:15,122 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.hbase-snapshot/.tmp/snaptb-1698451213243 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.hbase-snapshot/snaptb-1698451213243 2023-10-28 00:00:15,123 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1698451213243 of table testCompactionWithHFileLink completed 2023-10-28 00:00:15,123 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.hbase-snapshot/.tmp/snaptb-1698451213243 2023-10-28 00:00:15,123 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.hbase-snapshot/.tmp/snaptb-1698451213243 2023-10-28 00:00:15,123 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1698451213243 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1698451213295Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1698451214673 (+1378 ms)Consolidate snapshot: snaptb-1698451213243 at 1698451214674 (+1 ms)Loading Region manifests for snaptb-1698451213243 at 1698451214674Writing data manifest for snaptb-1698451213243 at 1698451214688 (+14 ms)Verifying snapshot: snaptb-1698451213243 at 1698451215098 (+410 ms)Snapshot snaptb-1698451213243 of table testCompactionWithHFileLink completed at 1698451215123 (+25 ms) 2023-10-28 00:00:15,124 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-10-28 00:00:15,125 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 1.8280 sec 2023-10-28 00:00:15,415 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-10-28 00:00:15,417 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=snaptb-1698451213243 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-10-28 00:00:15,417 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] snapshot.SnapshotManager(403): Snapshot '{ ss=snaptb-1698451213243 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-10-28 00:00:15,464 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-28 00:00:15,466 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-10-28 00:00:15,466 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-10-28 00:00:15,466 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-10-28 00:00:15,466 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:15,466 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-10-28 00:00:15,467 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-10-28 00:00:15,467 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:15,467 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-10-28 00:00:15,499 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:15,499 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:15,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:15,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:15,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-10-28 00:00:15,500 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:15,500 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-28 00:00:15,500 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:15,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-28 00:00:15,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-10-28 00:00:15,500 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-10-28 00:00:15,501 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-10-28 00:00:15,501 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-10-28 00:00:15,501 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-10-28 00:00:15,501 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:15,501 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-10-28 00:00:15,501 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:15,501 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. started... 2023-10-28 00:00:15,501 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:15,501 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-10-28 00:00:15,501 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-28 00:00:15,501 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:15,501 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. started... 2023-10-28 00:00:15,501 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4904326d2e05286728bb6d35b919e5bf 2/2 column families, dataSize=138 B heapSize=944 B 2023-10-28 00:00:15,501 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. started... 2023-10-28 00:00:15,501 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 795595fcd155993336d014aba73423ea 2/2 column families, dataSize=138 B heapSize=944 B 2023-10-28 00:00:15,501 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 6d789d75232206c2589efabacffe9d96 2/2 column families, dataSize=138 B heapSize=944 B 2023-10-28 00:00:15,920 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/.tmp/family1/587478d39b324adb930e20765d59b72c 2023-10-28 00:00:15,929 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/.tmp/family1/d6a83260e5de4ce5b675c2bef57b7421 2023-10-28 00:00:15,929 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 587478d39b324adb930e20765d59b72c 2023-10-28 00:00:15,933 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d6a83260e5de4ce5b675c2bef57b7421 2023-10-28 00:00:16,330 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/.tmp/family1/5d7a8c18f34f4385aa16e0a02cf2dea9 2023-10-28 00:00:16,348 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5d7a8c18f34f4385aa16e0a02cf2dea9 2023-10-28 00:00:16,357 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/.tmp/family2/c3950101249a449f8afbbd845deb05a3 2023-10-28 00:00:16,361 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c3950101249a449f8afbbd845deb05a3 2023-10-28 00:00:16,361 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/.tmp/family1/d6a83260e5de4ce5b675c2bef57b7421 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/d6a83260e5de4ce5b675c2bef57b7421 2023-10-28 00:00:16,365 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d6a83260e5de4ce5b675c2bef57b7421 2023-10-28 00:00:16,365 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/d6a83260e5de4ce5b675c2bef57b7421, entries=3, sequenceid=24, filesize=5.0 K 2023-10-28 00:00:16,366 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/.tmp/family2/c3950101249a449f8afbbd845deb05a3 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/c3950101249a449f8afbbd845deb05a3 2023-10-28 00:00:16,370 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c3950101249a449f8afbbd845deb05a3 2023-10-28 00:00:16,370 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/c3950101249a449f8afbbd845deb05a3, entries=1, sequenceid=24, filesize=5.0 K 2023-10-28 00:00:16,371 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 6d789d75232206c2589efabacffe9d96 in 870ms, sequenceid=24, compaction requested=true 2023-10-28 00:00:16,371 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 6d789d75232206c2589efabacffe9d96: 2023-10-28 00:00:16,371 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:16,371 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-10-28 00:00:16,757 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/.tmp/family2/095c37339514480da1ee37c0543dc1bf 2023-10-28 00:00:16,764 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 095c37339514480da1ee37c0543dc1bf 2023-10-28 00:00:16,766 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/.tmp/family1/587478d39b324adb930e20765d59b72c as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/587478d39b324adb930e20765d59b72c 2023-10-28 00:00:16,769 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/.tmp/family2/1cf3a9e64089412e90f0528ab5e873eb 2023-10-28 00:00:16,772 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 587478d39b324adb930e20765d59b72c 2023-10-28 00:00:16,772 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/587478d39b324adb930e20765d59b72c, entries=3, sequenceid=24, filesize=5.0 K 2023-10-28 00:00:16,773 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/.tmp/family2/095c37339514480da1ee37c0543dc1bf as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/095c37339514480da1ee37c0543dc1bf 2023-10-28 00:00:16,774 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1cf3a9e64089412e90f0528ab5e873eb 2023-10-28 00:00:16,775 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/.tmp/family1/5d7a8c18f34f4385aa16e0a02cf2dea9 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/5d7a8c18f34f4385aa16e0a02cf2dea9 2023-10-28 00:00:16,778 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 095c37339514480da1ee37c0543dc1bf 2023-10-28 00:00:16,778 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/095c37339514480da1ee37c0543dc1bf, entries=1, sequenceid=24, filesize=5.0 K 2023-10-28 00:00:16,779 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 795595fcd155993336d014aba73423ea in 1278ms, sequenceid=24, compaction requested=true 2023-10-28 00:00:16,779 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 795595fcd155993336d014aba73423ea: 2023-10-28 00:00:16,779 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:16,780 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-10-28 00:00:16,780 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5d7a8c18f34f4385aa16e0a02cf2dea9 2023-10-28 00:00:16,780 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/5d7a8c18f34f4385aa16e0a02cf2dea9, entries=3, sequenceid=24, filesize=5.0 K 2023-10-28 00:00:16,781 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/.tmp/family2/1cf3a9e64089412e90f0528ab5e873eb as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/1cf3a9e64089412e90f0528ab5e873eb 2023-10-28 00:00:16,786 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1cf3a9e64089412e90f0528ab5e873eb 2023-10-28 00:00:16,787 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/1cf3a9e64089412e90f0528ab5e873eb, entries=1, sequenceid=24, filesize=5.0 K 2023-10-28 00:00:16,787 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 4904326d2e05286728bb6d35b919e5bf in 1286ms, sequenceid=24, compaction requested=true 2023-10-28 00:00:16,788 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4904326d2e05286728bb6d35b919e5bf: 2023-10-28 00:00:16,788 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:16,788 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-10-28 00:00:16,788 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-10-28 00:00:16,788 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:16,788 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-10-28 00:00:16,788 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-10-28 00:00:16,837 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:16,837 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:16,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:16,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:16,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:16,838 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:16,838 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-28 00:00:16,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:16,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:16,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:16,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:16,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:16,844 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-10-28 00:00:16,844 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@67a9b73a[Count = 0] remaining members to acquire global barrier 2023-10-28 00:00:16,844 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-10-28 00:00:16,845 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:16,858 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:16,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:16,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:16,859 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-10-28 00:00:16,859 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-10-28 00:00:16,859 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-28 00:00:16,860 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:16,860 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-28 00:00:16,866 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:16,866 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-10-28 00:00:16,867 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:16,867 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-10-28 00:00:16,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:16,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:16,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:16,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:16,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:16,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:16,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:16,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:16,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:16,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:16,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-28 00:00:16,878 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-10-28 00:00:16,878 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-10-28 00:00:16,878 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-28 00:00:16,878 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-28 00:00:16,878 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-10-28 00:00:16,879 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-28 00:00:16,889 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:16,889 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:16,889 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:16,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:16,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:16,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:16,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:16,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:16,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:16,891 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:16,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:16,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:16,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:16,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:16,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:16,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:16,894 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:16,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:16,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:16,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-10-28 00:00:16,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:16,908 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:16,908 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:16,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:16,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:16,908 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:16,908 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-10-28 00:00:16,908 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-28 00:00:16,908 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:16,908 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-10-28 00:00:16,909 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-10-28 00:00:16,909 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-10-28 00:00:16,909 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-28 00:00:16,909 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-10-28 00:00:16,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:16,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:16,909 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:16,909 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:16,909 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-10-28 00:00:16,909 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-10-28 00:00:17,009 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-28 00:00:17,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-28 00:00:17,012 DEBUG [Listener at localhost/40295] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-10-28 00:00:17,013 DEBUG [Listener at localhost/40295] regionserver.HStore(1912): 4904326d2e05286728bb6d35b919e5bf/family1 is initiating major compaction (all files) 2023-10-28 00:00:17,013 INFO [Listener at localhost/40295] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-10-28 00:00:17,014 INFO [Listener at localhost/40295] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-10-28 00:00:17,016 INFO [Listener at localhost/40295] regionserver.HRegion(2259): Starting compaction of 4904326d2e05286728bb6d35b919e5bf/family1 in testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:17,016 INFO [Listener at localhost/40295] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/9e88a17ec01944c9a13e6ecf7608bf19, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/4765103779e64954835a94c7a00e92a0, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/9a188e707c0a4959b84ee66b9e2b7085, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/1f3a75cc77e74b86b83067b7864d8a22, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/5d7a8c18f34f4385aa16e0a02cf2dea9] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/.tmp, totalSize=26.7 K 2023-10-28 00:00:17,017 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 9e88a17ec01944c9a13e6ecf7608bf19, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698451205463 2023-10-28 00:00:17,018 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 4765103779e64954835a94c7a00e92a0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698451207431 2023-10-28 00:00:17,019 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 9a188e707c0a4959b84ee66b9e2b7085, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698451209412 2023-10-28 00:00:17,020 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 1f3a75cc77e74b86b83067b7864d8a22, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698451211340 2023-10-28 00:00:17,020 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 5d7a8c18f34f4385aa16e0a02cf2dea9, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-10-28 00:00:17,037 INFO [Listener at localhost/40295] throttle.PressureAwareThroughputController(145): 4904326d2e05286728bb6d35b919e5bf#family1#compaction#108 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-10-28 00:00:17,455 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20231028f6a4c92b90be45cb9195f9fe8162e99d_del 2023-10-28 00:00:17,455 INFO [Listener at localhost/40295] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231028f6a4c92b90be45cb9195f9fe8162e99d_del to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231028f6a4c92b90be45cb9195f9fe8162e99d_del 2023-10-28 00:00:17,922 DEBUG [regionserver/jenkins-hbase16:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.15384615384615385, tune throughput to 57.69 MB/second 2023-10-28 00:00:18,270 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 56557d83893b43cbab71c845495be2a5 2023-10-28 00:00:18,271 DEBUG [Listener at localhost/40295] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/.tmp/family1/56557d83893b43cbab71c845495be2a5 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/56557d83893b43cbab71c845495be2a5 2023-10-28 00:00:18,275 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 56557d83893b43cbab71c845495be2a5 2023-10-28 00:00:18,276 INFO [Listener at localhost/40295] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 4904326d2e05286728bb6d35b919e5bf/family1 of 4904326d2e05286728bb6d35b919e5bf into 56557d83893b43cbab71c845495be2a5(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-10-28 00:00:18,277 DEBUG [Listener at localhost/40295] regionserver.HRegion(2289): Compaction status journal for 4904326d2e05286728bb6d35b919e5bf: 2023-10-28 00:00:18,277 DEBUG [Listener at localhost/40295] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-10-28 00:00:18,277 DEBUG [Listener at localhost/40295] regionserver.HStore(1912): 4904326d2e05286728bb6d35b919e5bf/family2 is initiating major compaction (all files) 2023-10-28 00:00:18,277 INFO [Listener at localhost/40295] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-10-28 00:00:18,277 INFO [Listener at localhost/40295] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-10-28 00:00:18,277 INFO [Listener at localhost/40295] regionserver.HRegion(2259): Starting compaction of 4904326d2e05286728bb6d35b919e5bf/family2 in testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:18,277 INFO [Listener at localhost/40295] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/974f6211ca1740f2a34fe077aa91bdf2, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/2b1ba95eb4554512b2aa7f4f3bf684a7, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/01571f5665324edb985a32da4c4341a4, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/af19a53e844a4102a55235df865fc030, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/1cf3a9e64089412e90f0528ab5e873eb] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/.tmp, totalSize=25.4 K 2023-10-28 00:00:18,278 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 974f6211ca1740f2a34fe077aa91bdf2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698451205463 2023-10-28 00:00:18,278 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 2b1ba95eb4554512b2aa7f4f3bf684a7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698451207431 2023-10-28 00:00:18,278 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 01571f5665324edb985a32da4c4341a4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698451209412 2023-10-28 00:00:18,279 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting af19a53e844a4102a55235df865fc030, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698451211340 2023-10-28 00:00:18,279 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 1cf3a9e64089412e90f0528ab5e873eb, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-10-28 00:00:18,294 INFO [Listener at localhost/40295] throttle.PressureAwareThroughputController(145): 4904326d2e05286728bb6d35b919e5bf#family2#compaction#109 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-10-28 00:00:18,335 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/9e88a17ec01944c9a13e6ecf7608bf19, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/4765103779e64954835a94c7a00e92a0, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/9a188e707c0a4959b84ee66b9e2b7085, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/1f3a75cc77e74b86b83067b7864d8a22, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/5d7a8c18f34f4385aa16e0a02cf2dea9] to archive 2023-10-28 00:00:18,336 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:18,338 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/9e88a17ec01944c9a13e6ecf7608bf19 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/9e88a17ec01944c9a13e6ecf7608bf19 2023-10-28 00:00:18,340 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/4765103779e64954835a94c7a00e92a0 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/4765103779e64954835a94c7a00e92a0 2023-10-28 00:00:18,341 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/9a188e707c0a4959b84ee66b9e2b7085 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/9a188e707c0a4959b84ee66b9e2b7085 2023-10-28 00:00:18,342 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/1f3a75cc77e74b86b83067b7864d8a22 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/1f3a75cc77e74b86b83067b7864d8a22 2023-10-28 00:00:18,343 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/5d7a8c18f34f4385aa16e0a02cf2dea9 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/5d7a8c18f34f4385aa16e0a02cf2dea9 2023-10-28 00:00:18,713 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023102872c2f2c271b04250a482cc01e20fa15a_del 2023-10-28 00:00:18,714 INFO [Listener at localhost/40295] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023102872c2f2c271b04250a482cc01e20fa15a_del to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023102872c2f2c271b04250a482cc01e20fa15a_del 2023-10-28 00:00:19,538 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 479a91f53d9d43dcbdf8d8fa0df120d0 2023-10-28 00:00:19,539 DEBUG [Listener at localhost/40295] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/.tmp/family2/479a91f53d9d43dcbdf8d8fa0df120d0 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/479a91f53d9d43dcbdf8d8fa0df120d0 2023-10-28 00:00:19,542 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 479a91f53d9d43dcbdf8d8fa0df120d0 2023-10-28 00:00:19,543 INFO [Listener at localhost/40295] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 4904326d2e05286728bb6d35b919e5bf/family2 of 4904326d2e05286728bb6d35b919e5bf into 479a91f53d9d43dcbdf8d8fa0df120d0(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-10-28 00:00:19,543 DEBUG [Listener at localhost/40295] regionserver.HRegion(2289): Compaction status journal for 4904326d2e05286728bb6d35b919e5bf: 2023-10-28 00:00:19,544 DEBUG [Listener at localhost/40295] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-10-28 00:00:19,544 DEBUG [Listener at localhost/40295] regionserver.HStore(1912): 6d789d75232206c2589efabacffe9d96/family1 is initiating major compaction (all files) 2023-10-28 00:00:19,544 INFO [Listener at localhost/40295] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-10-28 00:00:19,544 INFO [Listener at localhost/40295] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-10-28 00:00:19,544 INFO [Listener at localhost/40295] regionserver.HRegion(2259): Starting compaction of 6d789d75232206c2589efabacffe9d96/family1 in testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:19,545 INFO [Listener at localhost/40295] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/7236d93111f746b6b44c723fc9e64af8, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/d39ca93432034ad29297bc42a5b0ea49, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/486a9ea7cf6e48928fb3a88650c0f7e6, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/c61d558f355843b0aec0e0a9e2c97536, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/d6a83260e5de4ce5b675c2bef57b7421] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/.tmp, totalSize=26.7 K 2023-10-28 00:00:19,545 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 7236d93111f746b6b44c723fc9e64af8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698451205462 2023-10-28 00:00:19,545 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting d39ca93432034ad29297bc42a5b0ea49, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698451207430 2023-10-28 00:00:19,546 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 486a9ea7cf6e48928fb3a88650c0f7e6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698451209411 2023-10-28 00:00:19,546 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting c61d558f355843b0aec0e0a9e2c97536, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698451211339 2023-10-28 00:00:19,546 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting d6a83260e5de4ce5b675c2bef57b7421, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-10-28 00:00:19,557 INFO [Listener at localhost/40295] throttle.PressureAwareThroughputController(145): 6d789d75232206c2589efabacffe9d96#family1#compaction#110 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-10-28 00:00:19,633 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/974f6211ca1740f2a34fe077aa91bdf2, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/2b1ba95eb4554512b2aa7f4f3bf684a7, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/01571f5665324edb985a32da4c4341a4, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/af19a53e844a4102a55235df865fc030, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/1cf3a9e64089412e90f0528ab5e873eb] to archive 2023-10-28 00:00:19,634 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:19,635 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/974f6211ca1740f2a34fe077aa91bdf2 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/974f6211ca1740f2a34fe077aa91bdf2 2023-10-28 00:00:19,637 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/2b1ba95eb4554512b2aa7f4f3bf684a7 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/2b1ba95eb4554512b2aa7f4f3bf684a7 2023-10-28 00:00:19,638 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/01571f5665324edb985a32da4c4341a4 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/01571f5665324edb985a32da4c4341a4 2023-10-28 00:00:19,639 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/af19a53e844a4102a55235df865fc030 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/af19a53e844a4102a55235df865fc030 2023-10-28 00:00:19,640 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/1cf3a9e64089412e90f0528ab5e873eb to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/1cf3a9e64089412e90f0528ab5e873eb 2023-10-28 00:00:19,972 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20231028186cbd3b947f4b179d7a18137433b2d7_del 2023-10-28 00:00:19,972 INFO [Listener at localhost/40295] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231028186cbd3b947f4b179d7a18137433b2d7_del to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231028186cbd3b947f4b179d7a18137433b2d7_del 2023-10-28 00:00:20,804 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for da418cf12cc24c7985572479eea26df2 2023-10-28 00:00:20,806 DEBUG [Listener at localhost/40295] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/.tmp/family1/da418cf12cc24c7985572479eea26df2 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/da418cf12cc24c7985572479eea26df2 2023-10-28 00:00:20,811 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for da418cf12cc24c7985572479eea26df2 2023-10-28 00:00:20,812 INFO [Listener at localhost/40295] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 6d789d75232206c2589efabacffe9d96/family1 of 6d789d75232206c2589efabacffe9d96 into da418cf12cc24c7985572479eea26df2(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-10-28 00:00:20,812 DEBUG [Listener at localhost/40295] regionserver.HRegion(2289): Compaction status journal for 6d789d75232206c2589efabacffe9d96: 2023-10-28 00:00:20,812 DEBUG [Listener at localhost/40295] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-10-28 00:00:20,812 DEBUG [Listener at localhost/40295] regionserver.HStore(1912): 6d789d75232206c2589efabacffe9d96/family2 is initiating major compaction (all files) 2023-10-28 00:00:20,812 INFO [Listener at localhost/40295] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-10-28 00:00:20,812 INFO [Listener at localhost/40295] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-10-28 00:00:20,812 INFO [Listener at localhost/40295] regionserver.HRegion(2259): Starting compaction of 6d789d75232206c2589efabacffe9d96/family2 in testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:20,812 INFO [Listener at localhost/40295] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/f854e3377f064dfc805f6e6239b2242e, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/69967cf4dac4418fb3e0be94fc433f04, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/2948c86034f5417187a0810c68ad2363, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/2f39466607184138b07dbe55159c1cf1, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/c3950101249a449f8afbbd845deb05a3] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/.tmp, totalSize=25.4 K 2023-10-28 00:00:20,813 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting f854e3377f064dfc805f6e6239b2242e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698451205462 2023-10-28 00:00:20,813 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 69967cf4dac4418fb3e0be94fc433f04, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698451207430 2023-10-28 00:00:20,813 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 2948c86034f5417187a0810c68ad2363, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698451209411 2023-10-28 00:00:20,814 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 2f39466607184138b07dbe55159c1cf1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698451211339 2023-10-28 00:00:20,814 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting c3950101249a449f8afbbd845deb05a3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-10-28 00:00:20,825 INFO [Listener at localhost/40295] throttle.PressureAwareThroughputController(145): 6d789d75232206c2589efabacffe9d96#family2#compaction#111 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-10-28 00:00:20,834 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/7236d93111f746b6b44c723fc9e64af8, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/d39ca93432034ad29297bc42a5b0ea49, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/486a9ea7cf6e48928fb3a88650c0f7e6, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/c61d558f355843b0aec0e0a9e2c97536, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/d6a83260e5de4ce5b675c2bef57b7421] to archive 2023-10-28 00:00:20,835 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:20,836 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/7236d93111f746b6b44c723fc9e64af8 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/7236d93111f746b6b44c723fc9e64af8 2023-10-28 00:00:20,837 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/d39ca93432034ad29297bc42a5b0ea49 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/d39ca93432034ad29297bc42a5b0ea49 2023-10-28 00:00:20,838 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/486a9ea7cf6e48928fb3a88650c0f7e6 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/486a9ea7cf6e48928fb3a88650c0f7e6 2023-10-28 00:00:20,839 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/c61d558f355843b0aec0e0a9e2c97536 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/c61d558f355843b0aec0e0a9e2c97536 2023-10-28 00:00:20,840 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/d6a83260e5de4ce5b675c2bef57b7421 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/d6a83260e5de4ce5b675c2bef57b7421 2023-10-28 00:00:21,249 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202310289230c344f9984111bf8bd37ab2b76f49_del 2023-10-28 00:00:21,249 INFO [Listener at localhost/40295] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202310289230c344f9984111bf8bd37ab2b76f49_del to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202310289230c344f9984111bf8bd37ab2b76f49_del 2023-10-28 00:00:22,098 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8c339dc3e6cd46efb3e49040d9c4414c 2023-10-28 00:00:22,100 DEBUG [Listener at localhost/40295] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/.tmp/family2/8c339dc3e6cd46efb3e49040d9c4414c as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/8c339dc3e6cd46efb3e49040d9c4414c 2023-10-28 00:00:22,110 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8c339dc3e6cd46efb3e49040d9c4414c 2023-10-28 00:00:22,112 INFO [Listener at localhost/40295] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 6d789d75232206c2589efabacffe9d96/family2 of 6d789d75232206c2589efabacffe9d96 into 8c339dc3e6cd46efb3e49040d9c4414c(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-10-28 00:00:22,113 DEBUG [Listener at localhost/40295] regionserver.HRegion(2289): Compaction status journal for 6d789d75232206c2589efabacffe9d96: 2023-10-28 00:00:22,113 DEBUG [Listener at localhost/40295] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-10-28 00:00:22,113 DEBUG [Listener at localhost/40295] regionserver.HStore(1912): 795595fcd155993336d014aba73423ea/family1 is initiating major compaction (all files) 2023-10-28 00:00:22,114 INFO [Listener at localhost/40295] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-10-28 00:00:22,114 INFO [Listener at localhost/40295] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-10-28 00:00:22,114 INFO [Listener at localhost/40295] regionserver.HRegion(2259): Starting compaction of 795595fcd155993336d014aba73423ea/family1 in testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:22,115 INFO [Listener at localhost/40295] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/dcf9afa91e314f178c59ced5a25fa944, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/30493979777049ce8ebd4cf3bc0cd5d5, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/2cb1c76d629141179060550b5919816b, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/1ac14e9e9ba54d8f8eae2adbb5a965e9, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/587478d39b324adb930e20765d59b72c] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/.tmp, totalSize=26.7 K 2023-10-28 00:00:22,115 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting dcf9afa91e314f178c59ced5a25fa944, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698451205461 2023-10-28 00:00:22,116 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 30493979777049ce8ebd4cf3bc0cd5d5, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698451207430 2023-10-28 00:00:22,116 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 2cb1c76d629141179060550b5919816b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698451209411 2023-10-28 00:00:22,117 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 1ac14e9e9ba54d8f8eae2adbb5a965e9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698451211339 2023-10-28 00:00:22,117 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 587478d39b324adb930e20765d59b72c, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-10-28 00:00:22,127 INFO [Listener at localhost/40295] throttle.PressureAwareThroughputController(145): 795595fcd155993336d014aba73423ea#family1#compaction#112 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-10-28 00:00:22,132 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/f854e3377f064dfc805f6e6239b2242e, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/69967cf4dac4418fb3e0be94fc433f04, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/2948c86034f5417187a0810c68ad2363, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/2f39466607184138b07dbe55159c1cf1, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/c3950101249a449f8afbbd845deb05a3] to archive 2023-10-28 00:00:22,133 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:22,134 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/f854e3377f064dfc805f6e6239b2242e to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/f854e3377f064dfc805f6e6239b2242e 2023-10-28 00:00:22,135 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/69967cf4dac4418fb3e0be94fc433f04 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/69967cf4dac4418fb3e0be94fc433f04 2023-10-28 00:00:22,137 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/2948c86034f5417187a0810c68ad2363 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/2948c86034f5417187a0810c68ad2363 2023-10-28 00:00:22,138 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/2f39466607184138b07dbe55159c1cf1 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/2f39466607184138b07dbe55159c1cf1 2023-10-28 00:00:22,139 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/c3950101249a449f8afbbd845deb05a3 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/c3950101249a449f8afbbd845deb05a3 2023-10-28 00:00:22,546 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20231028bd26b13ca8954f3bb4bf45072fc5b852_del 2023-10-28 00:00:22,547 INFO [Listener at localhost/40295] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231028bd26b13ca8954f3bb4bf45072fc5b852_del to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231028bd26b13ca8954f3bb4bf45072fc5b852_del 2023-10-28 00:00:23,380 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f44428b1317240da9c94292cf6ef2e25 2023-10-28 00:00:23,380 DEBUG [Listener at localhost/40295] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/.tmp/family1/f44428b1317240da9c94292cf6ef2e25 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/f44428b1317240da9c94292cf6ef2e25 2023-10-28 00:00:23,384 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f44428b1317240da9c94292cf6ef2e25 2023-10-28 00:00:23,385 INFO [Listener at localhost/40295] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 795595fcd155993336d014aba73423ea/family1 of 795595fcd155993336d014aba73423ea into f44428b1317240da9c94292cf6ef2e25(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-10-28 00:00:23,385 DEBUG [Listener at localhost/40295] regionserver.HRegion(2289): Compaction status journal for 795595fcd155993336d014aba73423ea: 2023-10-28 00:00:23,385 DEBUG [Listener at localhost/40295] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-10-28 00:00:23,386 DEBUG [Listener at localhost/40295] regionserver.HStore(1912): 795595fcd155993336d014aba73423ea/family2 is initiating major compaction (all files) 2023-10-28 00:00:23,386 INFO [Listener at localhost/40295] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-10-28 00:00:23,386 INFO [Listener at localhost/40295] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-10-28 00:00:23,386 INFO [Listener at localhost/40295] regionserver.HRegion(2259): Starting compaction of 795595fcd155993336d014aba73423ea/family2 in testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:23,386 INFO [Listener at localhost/40295] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/2bfcfdf09d734e5dac83a9d255bd7680, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/2083b1a0e27a4323afa5cc383672554b, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/4a1c8f8ec1b744a293ebf977fdd7f04d, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/1fc20602194041dda4e593b5943c1c21, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/095c37339514480da1ee37c0543dc1bf] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/.tmp, totalSize=25.4 K 2023-10-28 00:00:23,387 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 2bfcfdf09d734e5dac83a9d255bd7680, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698451205461 2023-10-28 00:00:23,387 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 2083b1a0e27a4323afa5cc383672554b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698451207430 2023-10-28 00:00:23,387 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 4a1c8f8ec1b744a293ebf977fdd7f04d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698451209411 2023-10-28 00:00:23,388 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 1fc20602194041dda4e593b5943c1c21, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698451211339 2023-10-28 00:00:23,388 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 095c37339514480da1ee37c0543dc1bf, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-10-28 00:00:23,397 INFO [Listener at localhost/40295] throttle.PressureAwareThroughputController(145): 795595fcd155993336d014aba73423ea#family2#compaction#113 average throughput is 1.05 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-10-28 00:00:23,429 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/dcf9afa91e314f178c59ced5a25fa944, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/30493979777049ce8ebd4cf3bc0cd5d5, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/2cb1c76d629141179060550b5919816b, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/1ac14e9e9ba54d8f8eae2adbb5a965e9, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/587478d39b324adb930e20765d59b72c] to archive 2023-10-28 00:00:23,430 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:23,433 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/dcf9afa91e314f178c59ced5a25fa944 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/dcf9afa91e314f178c59ced5a25fa944 2023-10-28 00:00:23,436 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/30493979777049ce8ebd4cf3bc0cd5d5 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/30493979777049ce8ebd4cf3bc0cd5d5 2023-10-28 00:00:23,438 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/2cb1c76d629141179060550b5919816b to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/2cb1c76d629141179060550b5919816b 2023-10-28 00:00:23,440 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/1ac14e9e9ba54d8f8eae2adbb5a965e9 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/1ac14e9e9ba54d8f8eae2adbb5a965e9 2023-10-28 00:00:23,442 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/587478d39b324adb930e20765d59b72c to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/587478d39b324adb930e20765d59b72c 2023-10-28 00:00:23,816 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202310287607fcf28bc14d318c273f72c4c326aa_del 2023-10-28 00:00:23,816 INFO [Listener at localhost/40295] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202310287607fcf28bc14d318c273f72c4c326aa_del to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202310287607fcf28bc14d318c273f72c4c326aa_del 2023-10-28 00:00:24,648 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9769baa6b5394d9cbea76a76732c0d3d 2023-10-28 00:00:24,650 DEBUG [Listener at localhost/40295] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/.tmp/family2/9769baa6b5394d9cbea76a76732c0d3d as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/9769baa6b5394d9cbea76a76732c0d3d 2023-10-28 00:00:24,655 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9769baa6b5394d9cbea76a76732c0d3d 2023-10-28 00:00:24,657 INFO [Listener at localhost/40295] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 795595fcd155993336d014aba73423ea/family2 of 795595fcd155993336d014aba73423ea into 9769baa6b5394d9cbea76a76732c0d3d(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-10-28 00:00:24,657 DEBUG [Listener at localhost/40295] regionserver.HRegion(2289): Compaction status journal for 795595fcd155993336d014aba73423ea: 2023-10-28 00:00:24,664 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-10-28 00:00:24,669 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-10-28 00:00:24,675 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-10-28 00:00:24,676 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-10-28 00:00:24,681 INFO [Listener at localhost/40295] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-10-28 00:00:24,718 INFO [Listener at localhost/40295] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-10-28 00:00:24,720 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20231028bd26b13ca8954f3bb4bf45072fc5b852_del 2023-10-28 00:00:24,723 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20231028186cbd3b947f4b179d7a18137433b2d7_del 2023-10-28 00:00:24,725 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20231028f6a4c92b90be45cb9195f9fe8162e99d_del 2023-10-28 00:00:24,725 INFO [Listener at localhost/40295] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-10-28 00:00:24,726 DEBUG [Listener at localhost/40295] zookeeper.ReadOnlyZKClient(139): Connect 0x6136aaf5 to 127.0.0.1:57369 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-10-28 00:00:24,737 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/2bfcfdf09d734e5dac83a9d255bd7680, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/2083b1a0e27a4323afa5cc383672554b, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/4a1c8f8ec1b744a293ebf977fdd7f04d, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/1fc20602194041dda4e593b5943c1c21, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/095c37339514480da1ee37c0543dc1bf] to archive 2023-10-28 00:00:24,738 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:24,739 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/2bfcfdf09d734e5dac83a9d255bd7680 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/2bfcfdf09d734e5dac83a9d255bd7680 2023-10-28 00:00:24,741 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/2083b1a0e27a4323afa5cc383672554b to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/2083b1a0e27a4323afa5cc383672554b 2023-10-28 00:00:24,742 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/4a1c8f8ec1b744a293ebf977fdd7f04d to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/4a1c8f8ec1b744a293ebf977fdd7f04d 2023-10-28 00:00:24,744 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/1fc20602194041dda4e593b5943c1c21 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/1fc20602194041dda4e593b5943c1c21 2023-10-28 00:00:24,745 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/095c37339514480da1ee37c0543dc1bf to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/095c37339514480da1ee37c0543dc1bf 2023-10-28 00:00:24,758 DEBUG [Listener at localhost/40295] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@67b34429, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-28 00:00:24,759 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20231028 for table='testCompactionWithHFileLink' and column='family1' 2023-10-28 00:00:24,759 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20231028 for table='testCompactionWithHFileLink' and column='family1' 2023-10-28 00:00:24,759 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20231028 for table='testCompactionWithHFileLink' and column='family1' 2023-10-28 00:00:25,234 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231028a588e4e5444a489f82291c78bd9852c5 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231028a588e4e5444a489f82291c78bd9852c5 2023-10-28 00:00:25,235 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231028 table='testCompactionWithHFileLink' column='family1' 2023-10-28 00:00:25,249 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-10-28 00:00:25,252 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:34928, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-10-28 00:00:25,258 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-10-28 00:00:25,260 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:44950, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-10-28 00:00:25,270 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231028/family1/968757f5add049eda9b811276c7367a2 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-10-28 00:00:25,271 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231028/family1/968757f5add049eda9b811276c7367a2}] 2023-10-28 00:00:25,275 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231028/family1/968757f5add049eda9b811276c7367a2 for inclusion in 6d789d75232206c2589efabacffe9d96/family1 2023-10-28 00:00:25,278 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-10-28 00:00:25,278 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(777): Region bounds: first=1 last=2 2023-10-28 00:00:25,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for 6d789d75232206c2589efabacffe9d96: 2023-10-28 00:00:25,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231028/family1/968757f5add049eda9b811276c7367a2 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__s57dtm85bgjnijmjarmbmqntf8m0a875sp2r0oj8k0sl6bkccvr8uv868r4p28v0/family1/968757f5add049eda9b811276c7367a2 2023-10-28 00:00:25,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__s57dtm85bgjnijmjarmbmqntf8m0a875sp2r0oj8k0sl6bkccvr8uv868r4p28v0/family1/968757f5add049eda9b811276c7367a2 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/5b977112009d4707a8fd1315c9a5947e_SeqId_29_ 2023-10-28 00:00:25,283 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__s57dtm85bgjnijmjarmbmqntf8m0a875sp2r0oj8k0sl6bkccvr8uv868r4p28v0/family1/968757f5add049eda9b811276c7367a2 into 6d789d75232206c2589efabacffe9d96/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/5b977112009d4707a8fd1315c9a5947e_SeqId_29_ - updating store file list. 2023-10-28 00:00:25,288 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/5b977112009d4707a8fd1315c9a5947e_SeqId_29_ into 6d789d75232206c2589efabacffe9d96/family1 2023-10-28 00:00:25,288 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__s57dtm85bgjnijmjarmbmqntf8m0a875sp2r0oj8k0sl6bkccvr8uv868r4p28v0/family1/968757f5add049eda9b811276c7367a2 into 6d789d75232206c2589efabacffe9d96/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/5b977112009d4707a8fd1315c9a5947e_SeqId_29_) 2023-10-28 00:00:25,288 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__s57dtm85bgjnijmjarmbmqntf8m0a875sp2r0oj8k0sl6bkccvr8uv868r4p28v0/family1/968757f5add049eda9b811276c7367a2 2023-10-28 00:00:25,291 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231028 table='testCompactionWithHFileLink' column='family1' 2023-10-28 00:00:25,292 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:25,294 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202310285de2655de28242baa74d1aa200b8e278 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202310285de2655de28242baa74d1aa200b8e278 2023-10-28 00:00:25,296 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231028b6ea31eec0664b918f7ee732a522247b to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231028b6ea31eec0664b918f7ee732a522247b 2023-10-28 00:00:25,298 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231028d89c12a20837409db30f53e7647e3333 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231028d89c12a20837409db30f53e7647e3333 2023-10-28 00:00:25,299 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231028f1ccb541978d4e818c1e1cd3c99edddd to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231028f1ccb541978d4e818c1e1cd3c99edddd 2023-10-28 00:00:25,299 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-10-28 00:00:25,299 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231028' 2023-10-28 00:00:25,300 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20231028 for table='testCompactionWithHFileLink' and column='family1' 2023-10-28 00:00:25,646 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202310283dbc91aadeee4679a75deb5d15326569 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202310283dbc91aadeee4679a75deb5d15326569 2023-10-28 00:00:25,647 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202310288e3cae6a65b340109f0df7d29b41f8a7 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202310288e3cae6a65b340109f0df7d29b41f8a7 2023-10-28 00:00:25,648 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231028 table='testCompactionWithHFileLink' column='family1' 2023-10-28 00:00:25,648 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231028 table='testCompactionWithHFileLink' column='family1' 2023-10-28 00:00:25,675 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231028/family1/d9b36beb55ed4660a6a717f2b26d4359 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-10-28 00:00:25,676 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231028/family1/d9b36beb55ed4660a6a717f2b26d4359}] 2023-10-28 00:00:25,677 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231028/family1/cd965b9c287a4d71833f10ce172a0ddd first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-10-28 00:00:25,677 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231028/family1/cd965b9c287a4d71833f10ce172a0ddd}] 2023-10-28 00:00:25,679 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231028/family1/d9b36beb55ed4660a6a717f2b26d4359 for inclusion in 4904326d2e05286728bb6d35b919e5bf/family1 2023-10-28 00:00:25,679 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231028/family1/cd965b9c287a4d71833f10ce172a0ddd for inclusion in 795595fcd155993336d014aba73423ea/family1 2023-10-28 00:00:25,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-10-28 00:00:25,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-10-28 00:00:25,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(777): Region bounds: first= last=1 2023-10-28 00:00:25,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43469] regionserver.HStore(777): Region bounds: first=2 last= 2023-10-28 00:00:25,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for 795595fcd155993336d014aba73423ea: 2023-10-28 00:00:25,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for 4904326d2e05286728bb6d35b919e5bf: 2023-10-28 00:00:25,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231028/family1/cd965b9c287a4d71833f10ce172a0ddd to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__468u8p455of7fdk5bogolt36bgsn1fsdqur7nqlq5lpef4em4jakcqcsurgfldg0/family1/cd965b9c287a4d71833f10ce172a0ddd 2023-10-28 00:00:25,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231028/family1/d9b36beb55ed4660a6a717f2b26d4359 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__o3u015nh45upgov1j87p6jav3eghbrt86jo9r9886ueuom8bmemsocalfhbonfrg/family1/d9b36beb55ed4660a6a717f2b26d4359 2023-10-28 00:00:25,685 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__468u8p455of7fdk5bogolt36bgsn1fsdqur7nqlq5lpef4em4jakcqcsurgfldg0/family1/cd965b9c287a4d71833f10ce172a0ddd as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/8843aa9c1f8445ff92f245256f51e630_SeqId_29_ 2023-10-28 00:00:25,685 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__o3u015nh45upgov1j87p6jav3eghbrt86jo9r9886ueuom8bmemsocalfhbonfrg/family1/d9b36beb55ed4660a6a717f2b26d4359 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/5a967d42418f42ec98b8b49e22c54f2d_SeqId_29_ 2023-10-28 00:00:25,685 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__468u8p455of7fdk5bogolt36bgsn1fsdqur7nqlq5lpef4em4jakcqcsurgfldg0/family1/cd965b9c287a4d71833f10ce172a0ddd into 795595fcd155993336d014aba73423ea/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/8843aa9c1f8445ff92f245256f51e630_SeqId_29_ - updating store file list. 2023-10-28 00:00:25,685 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__o3u015nh45upgov1j87p6jav3eghbrt86jo9r9886ueuom8bmemsocalfhbonfrg/family1/d9b36beb55ed4660a6a717f2b26d4359 into 4904326d2e05286728bb6d35b919e5bf/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/5a967d42418f42ec98b8b49e22c54f2d_SeqId_29_ - updating store file list. 2023-10-28 00:00:25,688 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/8843aa9c1f8445ff92f245256f51e630_SeqId_29_ into 795595fcd155993336d014aba73423ea/family1 2023-10-28 00:00:25,688 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__468u8p455of7fdk5bogolt36bgsn1fsdqur7nqlq5lpef4em4jakcqcsurgfldg0/family1/cd965b9c287a4d71833f10ce172a0ddd into 795595fcd155993336d014aba73423ea/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/8843aa9c1f8445ff92f245256f51e630_SeqId_29_) 2023-10-28 00:00:25,689 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/5a967d42418f42ec98b8b49e22c54f2d_SeqId_29_ into 4904326d2e05286728bb6d35b919e5bf/family1 2023-10-28 00:00:25,689 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__o3u015nh45upgov1j87p6jav3eghbrt86jo9r9886ueuom8bmemsocalfhbonfrg/family1/d9b36beb55ed4660a6a717f2b26d4359 into 4904326d2e05286728bb6d35b919e5bf/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/5a967d42418f42ec98b8b49e22c54f2d_SeqId_29_) 2023-10-28 00:00:25,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__468u8p455of7fdk5bogolt36bgsn1fsdqur7nqlq5lpef4em4jakcqcsurgfldg0/family1/cd965b9c287a4d71833f10ce172a0ddd 2023-10-28 00:00:25,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__o3u015nh45upgov1j87p6jav3eghbrt86jo9r9886ueuom8bmemsocalfhbonfrg/family1/d9b36beb55ed4660a6a717f2b26d4359 2023-10-28 00:00:25,691 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231028 table='testCompactionWithHFileLink' column='family1' 2023-10-28 00:00:25,691 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231028 table='testCompactionWithHFileLink' column='family1' 2023-10-28 00:00:25,691 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:25,691 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:25,693 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202310283b7c07e40bf24d94844e40d254645d1f to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202310283b7c07e40bf24d94844e40d254645d1f 2023-10-28 00:00:25,693 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202310284c7a22bee8b540ad908824b1f36d2ef3 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202310284c7a22bee8b540ad908824b1f36d2ef3 2023-10-28 00:00:25,693 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023102895f8921a8d4d4d278d029ad418c53fc9 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023102895f8921a8d4d4d278d029ad418c53fc9 2023-10-28 00:00:25,693 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231028625ac864940d4814bc5b572d46243862 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231028625ac864940d4814bc5b572d46243862 2023-10-28 00:00:25,694 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202310289c1bbedd5cfa43539015c979c7f943db to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202310289c1bbedd5cfa43539015c979c7f943db 2023-10-28 00:00:25,694 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023102872405d973040401f805bd31e54b32525 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023102872405d973040401f805bd31e54b32525 2023-10-28 00:00:25,695 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202310289d878da8c1b44dcc97673584130fa660 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202310289d878da8c1b44dcc97673584130fa660 2023-10-28 00:00:25,695 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-10-28 00:00:25,695 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202310289cf51282aea94d7d81e7ee5e8cb74330 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202310289cf51282aea94d7d81e7ee5e8cb74330 2023-10-28 00:00:25,695 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231028' 2023-10-28 00:00:25,695 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-10-28 00:00:25,695 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231028' 2023-10-28 00:00:25,695 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20231028 for table='testCompactionWithHFileLink' and column='family1' 2023-10-28 00:00:25,695 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20231028 for table='testCompactionWithHFileLink' and column='family1' 2023-10-28 00:00:25,696 INFO [Listener at localhost/40295] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-10-28 00:00:25,696 INFO [Listener at localhost/40295] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-10-28 00:00:25,696 INFO [Listener at localhost/40295] compactions.PartitionedMobCompactor(367): [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231028bd26b13ca8954f3bb4bf45072fc5b852_del] 2023-10-28 00:00:25,696 DEBUG [Listener at localhost/40295] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:25,697 DEBUG [Listener at localhost/40295] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231028bd26b13ca8954f3bb4bf45072fc5b852_del to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231028bd26b13ca8954f3bb4bf45072fc5b852_del 2023-10-28 00:00:25,697 INFO [Listener at localhost/40295] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-10-28 00:00:25,697 INFO [Listener at localhost/40295] compactions.PartitionedMobCompactor(367): [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231028186cbd3b947f4b179d7a18137433b2d7_del] 2023-10-28 00:00:25,697 DEBUG [Listener at localhost/40295] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:25,698 DEBUG [Listener at localhost/40295] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231028186cbd3b947f4b179d7a18137433b2d7_del to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231028186cbd3b947f4b179d7a18137433b2d7_del 2023-10-28 00:00:25,699 INFO [Listener at localhost/40295] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-10-28 00:00:25,699 INFO [Listener at localhost/40295] compactions.PartitionedMobCompactor(367): [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231028f6a4c92b90be45cb9195f9fe8162e99d_del] 2023-10-28 00:00:25,699 DEBUG [Listener at localhost/40295] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:25,700 DEBUG [Listener at localhost/40295] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231028f6a4c92b90be45cb9195f9fe8162e99d_del to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231028f6a4c92b90be45cb9195f9fe8162e99d_del 2023-10-28 00:00:25,700 INFO [Listener at localhost/40295] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-10-28 00:00:25,703 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-10-28 00:00:25,706 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-10-28 00:00:25,709 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-10-28 00:00:25,710 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-10-28 00:00:25,718 INFO [Listener at localhost/40295] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-10-28 00:00:25,718 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.HMaster$11(2418): Client=jenkins//188.40.62.62 disable testCompactionWithHFileLink 2023-10-28 00:00:25,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure2.ProcedureExecutor(1028): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-10-28 00:00:25,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-10-28 00:00:25,722 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451225722"}]},"ts":"1698451225722"} 2023-10-28 00:00:25,723 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-10-28 00:00:25,757 INFO [PEWorker-4] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-10-28 00:00:25,758 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=795595fcd155993336d014aba73423ea, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6d789d75232206c2589efabacffe9d96, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=4904326d2e05286728bb6d35b919e5bf, UNASSIGN}] 2023-10-28 00:00:25,759 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=4904326d2e05286728bb6d35b919e5bf, UNASSIGN 2023-10-28 00:00:25,760 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=795595fcd155993336d014aba73423ea, UNASSIGN 2023-10-28 00:00:25,760 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6d789d75232206c2589efabacffe9d96, UNASSIGN 2023-10-28 00:00:25,760 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=4904326d2e05286728bb6d35b919e5bf, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:25,760 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=795595fcd155993336d014aba73423ea, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:25,760 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698451225760"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451225760"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451225760"}]},"ts":"1698451225760"} 2023-10-28 00:00:25,761 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698451225760"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451225760"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451225760"}]},"ts":"1698451225760"} 2023-10-28 00:00:25,761 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=6d789d75232206c2589efabacffe9d96, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:25,761 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1698451225761"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451225761"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451225761"}]},"ts":"1698451225761"} 2023-10-28 00:00:25,762 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=92, ppid=91, state=RUNNABLE; CloseRegionProcedure 4904326d2e05286728bb6d35b919e5bf, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-28 00:00:25,763 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=93, ppid=89, state=RUNNABLE; CloseRegionProcedure 795595fcd155993336d014aba73423ea, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-28 00:00:25,763 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=94, ppid=90, state=RUNNABLE; CloseRegionProcedure 6d789d75232206c2589efabacffe9d96, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-28 00:00:25,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-10-28 00:00:25,914 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:25,915 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(111): Close 4904326d2e05286728bb6d35b919e5bf 2023-10-28 00:00:25,915 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing 4904326d2e05286728bb6d35b919e5bf, disabling compactions & flushes 2023-10-28 00:00:25,915 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:25,915 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:25,915 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. after waiting 0 ms 2023-10-28 00:00:25,916 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:25,930 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-10-28 00:00:25,932 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:25,932 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for 4904326d2e05286728bb6d35b919e5bf: 2023-10-28 00:00:25,934 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(149): Closed 4904326d2e05286728bb6d35b919e5bf 2023-10-28 00:00:25,934 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(111): Close 795595fcd155993336d014aba73423ea 2023-10-28 00:00:25,934 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing 795595fcd155993336d014aba73423ea, disabling compactions & flushes 2023-10-28 00:00:25,934 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:25,934 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:25,934 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. after waiting 0 ms 2023-10-28 00:00:25,934 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:25,934 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=4904326d2e05286728bb6d35b919e5bf, regionState=CLOSED 2023-10-28 00:00:25,935 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698451225934"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451225934"}]},"ts":"1698451225934"} 2023-10-28 00:00:25,942 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=92, resume processing ppid=91 2023-10-28 00:00:25,942 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=92, ppid=91, state=SUCCESS; CloseRegionProcedure 4904326d2e05286728bb6d35b919e5bf, server=jenkins-hbase16.apache.org,43469,1698451096272 in 179 msec 2023-10-28 00:00:25,943 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=4904326d2e05286728bb6d35b919e5bf, UNASSIGN in 184 msec 2023-10-28 00:00:25,943 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-10-28 00:00:25,944 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:25,944 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for 795595fcd155993336d014aba73423ea: 2023-10-28 00:00:25,946 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(149): Closed 795595fcd155993336d014aba73423ea 2023-10-28 00:00:25,946 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(111): Close 6d789d75232206c2589efabacffe9d96 2023-10-28 00:00:25,946 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing 6d789d75232206c2589efabacffe9d96, disabling compactions & flushes 2023-10-28 00:00:25,946 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:25,946 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:25,946 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. after waiting 0 ms 2023-10-28 00:00:25,946 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:25,946 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=795595fcd155993336d014aba73423ea, regionState=CLOSED 2023-10-28 00:00:25,946 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698451225946"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451225946"}]},"ts":"1698451225946"} 2023-10-28 00:00:25,950 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=93, resume processing ppid=89 2023-10-28 00:00:25,950 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=93, ppid=89, state=SUCCESS; CloseRegionProcedure 795595fcd155993336d014aba73423ea, server=jenkins-hbase16.apache.org,43469,1698451096272 in 187 msec 2023-10-28 00:00:25,951 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-10-28 00:00:25,951 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=795595fcd155993336d014aba73423ea, UNASSIGN in 192 msec 2023-10-28 00:00:25,952 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:25,952 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for 6d789d75232206c2589efabacffe9d96: 2023-10-28 00:00:25,953 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(149): Closed 6d789d75232206c2589efabacffe9d96 2023-10-28 00:00:25,954 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=6d789d75232206c2589efabacffe9d96, regionState=CLOSED 2023-10-28 00:00:25,954 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1698451225954"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451225954"}]},"ts":"1698451225954"} 2023-10-28 00:00:25,957 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=94, resume processing ppid=90 2023-10-28 00:00:25,957 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=94, ppid=90, state=SUCCESS; CloseRegionProcedure 6d789d75232206c2589efabacffe9d96, server=jenkins-hbase16.apache.org,43469,1698451096272 in 192 msec 2023-10-28 00:00:25,958 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=90, resume processing ppid=88 2023-10-28 00:00:25,958 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6d789d75232206c2589efabacffe9d96, UNASSIGN in 199 msec 2023-10-28 00:00:25,959 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451225959"}]},"ts":"1698451225959"} 2023-10-28 00:00:25,960 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-10-28 00:00:25,965 INFO [PEWorker-2] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-10-28 00:00:25,967 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 247 msec 2023-10-28 00:00:26,024 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-10-28 00:00:26,024 INFO [Listener at localhost/40295] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-10-28 00:00:26,038 INFO [Listener at localhost/40295] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1698451213243-1698451226038 2023-10-28 00:00:26,038 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1583): Client=jenkins//188.40.62.62 snapshot request for:{ ss=hbase-failsafe-snaptb-1698451213243-1698451226038 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-10-28 00:00:26,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1698451226038 (current time:1698451226038). 2023-10-28 00:00:26,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-10-28 00:00:26,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] zookeeper.ReadOnlyZKClient(139): Connect 0x668efd4a to 127.0.0.1:57369 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-10-28 00:00:26,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5e8c8c6b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-28 00:00:26,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-10-28 00:00:26,054 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:34932, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-10-28 00:00:26,054 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x668efd4a to 127.0.0.1:57369 2023-10-28 00:00:26,054 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] ipc.AbstractRpcClient(494): Stopping rpc client 2023-10-28 00:00:26,055 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] snapshot.SnapshotManager(637): No existing snapshot, attempting snapshot... 2023-10-28 00:00:26,055 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] snapshot.SnapshotManager(695): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1698451213243-1698451226038 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-10-28 00:00:26,058 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] snapshot.SnapshotManager(519): Unable to retrieve ACL status for path: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.hbase-snapshot, current working dir path: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1698451213243-1698451226038 org.apache.hadoop.hdfs.protocol.AclException: The ACL operation has been rejected. Support for ACLs has been disabled by setting dfs.namenode.acls.enabled to false. at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.checkAclsConfigFlag(FSDirAclOp.java:197) at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.getAclStatus(FSDirAclOp.java:150) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:6744) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getAclStatus(NameNodeRpcServer.java:2065) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getAclStatus(ClientNamenodeProtocolServerSideTranslatorPB.java:1378) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.getAclStatus(DFSClient.java:2736) at org.apache.hadoop.hdfs.DistributedFileSystem$51.doCall(DistributedFileSystem.java:2294) at org.apache.hadoop.hdfs.DistributedFileSystem$51.doCall(DistributedFileSystem.java:2291) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getAclStatus(DistributedFileSystem.java:2291) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.updateWorkingDirAclsIfRequired(SnapshotManager.java:517) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.prepareToTakeSnapshot(SnapshotManager.java:490) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.snapshotDisabledTable(SnapshotManager.java:541) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.takeSnapshotInternal(SnapshotManager.java:698) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.takeSnapshot(SnapshotManager.java:623) at org.apache.hadoop.hbase.master.MasterRpcServices.snapshot(MasterRpcServices.java:1588) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.AclException): The ACL operation has been rejected. Support for ACLs has been disabled by setting dfs.namenode.acls.enabled to false. at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.checkAclsConfigFlag(FSDirAclOp.java:197) at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.getAclStatus(FSDirAclOp.java:150) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:6744) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getAclStatus(NameNodeRpcServer.java:2065) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getAclStatus(ClientNamenodeProtocolServerSideTranslatorPB.java:1378) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1540) at org.apache.hadoop.ipc.Client.call(Client.java:1486) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy29.getAclStatus(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getAclStatus(ClientNamenodeProtocolTranslatorPB.java:1385) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy30.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.getAclStatus(DFSClient.java:2734) ... 15 more 2023-10-28 00:00:26,061 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure2.ProcedureExecutor(1028): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-10-28 00:00:26,062 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-10-28 00:00:26,063 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1698451826063 2023-10-28 00:00:26,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] snapshot.SnapshotManager(700): Started snapshot: { ss=hbase-failsafe-snaptb-1698451213243-1698451226038 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-10-28 00:00:26,063 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1698451213243-1698451226038 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-10-28 00:00:26,064 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1698451213243-1698451226038 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-10-28 00:00:26,064 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-10-28 00:00:26,164 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-10-28 00:00:26,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1698451213243-1698451226038 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-10-28 00:00:26,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] snapshot.SnapshotManager(406): Snapshoting '{ ss=hbase-failsafe-snaptb-1698451213243-1698451226038 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-10-28 00:00:26,168 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-10-28 00:00:26,368 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-10-28 00:00:26,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1698451213243-1698451226038 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-10-28 00:00:26,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] snapshot.SnapshotManager(406): Snapshoting '{ ss=hbase-failsafe-snaptb-1698451213243-1698451226038 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-10-28 00:00:26,370 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-10-28 00:00:26,476 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1698451213243-1698451226038 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-10-28 00:00:26,481 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-10-28 00:00:26,481 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-10-28 00:00:26,481 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-10-28 00:00:26,481 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-10-28 00:00:26,481 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-10-28 00:00:26,482 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-10-28 00:00:26,482 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-10-28 00:00:26,482 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-10-28 00:00:26,484 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/8843aa9c1f8445ff92f245256f51e630_SeqId_29_, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/f44428b1317240da9c94292cf6ef2e25] hfiles 2023-10-28 00:00:26,484 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/8843aa9c1f8445ff92f245256f51e630_SeqId_29_ 2023-10-28 00:00:26,484 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/56557d83893b43cbab71c845495be2a5, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/5a967d42418f42ec98b8b49e22c54f2d_SeqId_29_] hfiles 2023-10-28 00:00:26,484 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/f44428b1317240da9c94292cf6ef2e25 2023-10-28 00:00:26,484 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/56557d83893b43cbab71c845495be2a5 2023-10-28 00:00:26,484 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/5b977112009d4707a8fd1315c9a5947e_SeqId_29_, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/da418cf12cc24c7985572479eea26df2] hfiles 2023-10-28 00:00:26,484 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/5b977112009d4707a8fd1315c9a5947e_SeqId_29_ 2023-10-28 00:00:26,484 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/5a967d42418f42ec98b8b49e22c54f2d_SeqId_29_ 2023-10-28 00:00:26,484 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/da418cf12cc24c7985572479eea26df2 2023-10-28 00:00:26,485 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231028a588e4e5444a489f82291c78bd9852c5, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202310288e3cae6a65b340109f0df7d29b41f8a7, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202310283dbc91aadeee4679a75deb5d15326569] hfiles 2023-10-28 00:00:26,485 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231028a588e4e5444a489f82291c78bd9852c5 2023-10-28 00:00:26,485 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202310288e3cae6a65b340109f0df7d29b41f8a7 2023-10-28 00:00:26,485 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202310283dbc91aadeee4679a75deb5d15326569 2023-10-28 00:00:26,485 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/9769baa6b5394d9cbea76a76732c0d3d] hfiles 2023-10-28 00:00:26,485 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/9769baa6b5394d9cbea76a76732c0d3d 2023-10-28 00:00:26,485 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/8c339dc3e6cd46efb3e49040d9c4414c] hfiles 2023-10-28 00:00:26,485 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/8c339dc3e6cd46efb3e49040d9c4414c 2023-10-28 00:00:26,487 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/479a91f53d9d43dcbdf8d8fa0df120d0] hfiles 2023-10-28 00:00:26,487 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/479a91f53d9d43dcbdf8d8fa0df120d0 2023-10-28 00:00:26,494 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202310282f8d01bde2ac4b5b9f8db3d7eb0d5a77, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023102869445068db394b3ebed604128d1a21e2, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023102881dc0532d5044ef7b13d871661b0cdfd, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202310289230c344f9984111bf8bd37ab2b76f49_del, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231028ebb1400b1e1248c28157257d7b397f33, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023102821446b44bbb94f49ab9f91f9fa5050d6, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023102872c2f2c271b04250a482cc01e20fa15a_del, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023102898ec894991c44e9a840d313061645c04, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231028a720d1ccf6fe4f3ebc304645dfd29979, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231028f817b6d57d2c40f69a6e2c74a2770771, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202310287607fcf28bc14d318c273f72c4c326aa_del, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231028a5ae3e73702f43c2bae66963f508b295, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231028b039874f3c504d3dbaa25299e35b785a, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231028c5f21f48281a40bbb55088320038093a, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231028ceece078db4740dc8c8b77e7268b7665] hfiles 2023-10-28 00:00:26,495 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202310282f8d01bde2ac4b5b9f8db3d7eb0d5a77 2023-10-28 00:00:26,495 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023102869445068db394b3ebed604128d1a21e2 2023-10-28 00:00:26,495 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023102881dc0532d5044ef7b13d871661b0cdfd 2023-10-28 00:00:26,495 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202310289230c344f9984111bf8bd37ab2b76f49_del 2023-10-28 00:00:26,495 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231028ebb1400b1e1248c28157257d7b397f33 2023-10-28 00:00:26,495 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023102821446b44bbb94f49ab9f91f9fa5050d6 2023-10-28 00:00:26,495 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023102872c2f2c271b04250a482cc01e20fa15a_del 2023-10-28 00:00:26,495 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023102898ec894991c44e9a840d313061645c04 2023-10-28 00:00:26,495 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231028a720d1ccf6fe4f3ebc304645dfd29979 2023-10-28 00:00:26,495 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231028f817b6d57d2c40f69a6e2c74a2770771 2023-10-28 00:00:26,495 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202310287607fcf28bc14d318c273f72c4c326aa_del 2023-10-28 00:00:26,495 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231028a5ae3e73702f43c2bae66963f508b295 2023-10-28 00:00:26,495 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231028b039874f3c504d3dbaa25299e35b785a 2023-10-28 00:00:26,495 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231028c5f21f48281a40bbb55088320038093a 2023-10-28 00:00:26,495 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231028ceece078db4740dc8c8b77e7268b7665 2023-10-28 00:00:26,671 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-10-28 00:00:26,672 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1698451213243-1698451226038 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-10-28 00:00:26,672 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] snapshot.SnapshotManager(406): Snapshoting '{ ss=hbase-failsafe-snaptb-1698451213243-1698451226038 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-10-28 00:00:26,672 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-10-28 00:00:26,906 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1698451213243-1698451226038 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-10-28 00:00:26,906 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1698451213243-1698451226038 2023-10-28 00:00:26,908 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1698451213243-1698451226038 2023-10-28 00:00:27,173 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-10-28 00:00:27,175 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1698451213243-1698451226038 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-10-28 00:00:27,175 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] snapshot.SnapshotManager(406): Snapshoting '{ ss=hbase-failsafe-snaptb-1698451213243-1698451226038 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-10-28 00:00:27,175 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-10-28 00:00:27,360 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1698451213243-1698451226038 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.hbase-snapshot/hbase-failsafe-snaptb-1698451213243-1698451226038 2023-10-28 00:00:27,360 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1698451213243-1698451226038 of table testCompactionWithHFileLink completed 2023-10-28 00:00:27,360 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1698451213243-1698451226038 2023-10-28 00:00:27,361 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1698451213243-1698451226038 2023-10-28 00:00:27,361 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1698451213243-1698451226038 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1698451226064Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1698451213243-1698451226038 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1698451226476 (+412 ms)Consolidate snapshot: hbase-failsafe-snaptb-1698451213243-1698451226038 at 1698451226906 (+430 ms)Loading Region manifests for hbase-failsafe-snaptb-1698451213243-1698451226038 at 1698451226907 (+1 ms)Writing data manifest for hbase-failsafe-snaptb-1698451213243-1698451226038 at 1698451226913 (+6 ms)Verifying snapshot: hbase-failsafe-snaptb-1698451213243-1698451226038 at 1698451227337 (+424 ms)Snapshot hbase-failsafe-snaptb-1698451213243-1698451226038 of table testCompactionWithHFileLink completed at 1698451227360 (+23 ms) 2023-10-28 00:00:27,362 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-10-28 00:00:27,363 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 1.3020 sec 2023-10-28 00:00:28,176 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-10-28 00:00:28,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1698451213243-1698451226038 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-10-28 00:00:28,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] snapshot.SnapshotManager(403): Snapshot '{ ss=hbase-failsafe-snaptb-1698451213243-1698451226038 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-10-28 00:00:28,224 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure2.ProcedureExecutor(1028): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1698451213243" table: "testCompactionWithHFileLink" creation_time: 1698451213255 type: FLUSH version: 2 ttl: 0 ) 2023-10-28 00:00:28,224 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] snapshot.SnapshotManager(916): Restore snapshot=snaptb-1698451213243 as table=testCompactionWithHFileLink 2023-10-28 00:00:28,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-10-28 00:00:28,328 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-10-28 00:00:28,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-10-28 00:00:28,636 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-10-28 00:00:28,639 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-10-28 00:00:28,639 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-10-28 00:00:28,641 INFO [PEWorker-1] procedure.RestoreSnapshotProcedure(384): Starting restore snapshot={ ss=snaptb-1698451213243 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-10-28 00:00:28,648 INFO [PEWorker-1] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1698451213243" table: "testCompactionWithHFileLink" creation_time: 1698451213255 type: FLUSH version: 2 ttl: 0 2023-10-28 00:00:28,649 DEBUG [PEWorker-1] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink 2023-10-28 00:00:28,656 DEBUG [PEWorker-1] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-10-28 00:00:28,659 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202310288e3cae6a65b340109f0df7d29b41f8a7 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202310288e3cae6a65b340109f0df7d29b41f8a7 2023-10-28 00:00:28,660 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231028a588e4e5444a489f82291c78bd9852c5 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231028a588e4e5444a489f82291c78bd9852c5 2023-10-28 00:00:28,661 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202310283dbc91aadeee4679a75deb5d15326569 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202310283dbc91aadeee4679a75deb5d15326569 2023-10-28 00:00:28,661 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202310285de2655de28242baa74d1aa200b8e278 of snapshot=snaptb-1698451213243 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-10-28 00:00:28,664 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20231028b6ea31eec0664b918f7ee732a522247b of snapshot=snaptb-1698451213243 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-10-28 00:00:28,666 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20231028d89c12a20837409db30f53e7647e3333 of snapshot=snaptb-1698451213243 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-10-28 00:00:28,670 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20231028f1ccb541978d4e818c1e1cd3c99edddd of snapshot=snaptb-1698451213243 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-10-28 00:00:28,674 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202310283b7c07e40bf24d94844e40d254645d1f of snapshot=snaptb-1698451213243 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-10-28 00:00:28,678 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c2023102895f8921a8d4d4d278d029ad418c53fc9 of snapshot=snaptb-1698451213243 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-10-28 00:00:28,682 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202310289c1bbedd5cfa43539015c979c7f943db of snapshot=snaptb-1698451213243 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-10-28 00:00:28,687 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202310289d878da8c1b44dcc97673584130fa660 of snapshot=snaptb-1698451213243 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-10-28 00:00:28,692 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202310284c7a22bee8b540ad908824b1f36d2ef3 of snapshot=snaptb-1698451213243 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-10-28 00:00:28,694 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20231028625ac864940d4814bc5b572d46243862 of snapshot=snaptb-1698451213243 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-10-28 00:00:28,697 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e2023102872405d973040401f805bd31e54b32525 of snapshot=snaptb-1698451213243 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-10-28 00:00:28,700 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202310289cf51282aea94d7d81e7ee5e8cb74330 of snapshot=snaptb-1698451213243 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-10-28 00:00:28,705 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202310289230c344f9984111bf8bd37ab2b76f49_del to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202310289230c344f9984111bf8bd37ab2b76f49_del 2023-10-28 00:00:28,707 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202310287607fcf28bc14d318c273f72c4c326aa_del to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202310287607fcf28bc14d318c273f72c4c326aa_del 2023-10-28 00:00:28,708 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023102872c2f2c271b04250a482cc01e20fa15a_del to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023102872c2f2c271b04250a482cc01e20fa15a_del 2023-10-28 00:00:28,708 INFO [PEWorker-1] snapshot.RestoreSnapshotHelper(223): region to restore: 4904326d2e05286728bb6d35b919e5bf 2023-10-28 00:00:28,708 INFO [PEWorker-1] snapshot.RestoreSnapshotHelper(223): region to restore: 6d789d75232206c2589efabacffe9d96 2023-10-28 00:00:28,708 INFO [PEWorker-1] snapshot.RestoreSnapshotHelper(223): region to restore: 795595fcd155993336d014aba73423ea 2023-10-28 00:00:28,713 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/5a967d42418f42ec98b8b49e22c54f2d_SeqId_29_ to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/5a967d42418f42ec98b8b49e22c54f2d_SeqId_29_ 2023-10-28 00:00:28,713 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/da418cf12cc24c7985572479eea26df2 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/da418cf12cc24c7985572479eea26df2 2023-10-28 00:00:28,713 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/8843aa9c1f8445ff92f245256f51e630_SeqId_29_ to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/8843aa9c1f8445ff92f245256f51e630_SeqId_29_ 2023-10-28 00:00:28,714 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/56557d83893b43cbab71c845495be2a5 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/56557d83893b43cbab71c845495be2a5 2023-10-28 00:00:28,715 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9e88a17ec01944c9a13e6ecf7608bf19 of snapshot=snaptb-1698451213243 to region=4904326d2e05286728bb6d35b919e5bf table=testCompactionWithHFileLink 2023-10-28 00:00:28,715 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/5b977112009d4707a8fd1315c9a5947e_SeqId_29_ to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/5b977112009d4707a8fd1315c9a5947e_SeqId_29_ 2023-10-28 00:00:28,715 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 7236d93111f746b6b44c723fc9e64af8 of snapshot=snaptb-1698451213243 to region=6d789d75232206c2589efabacffe9d96 table=testCompactionWithHFileLink 2023-10-28 00:00:28,715 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/f44428b1317240da9c94292cf6ef2e25 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/f44428b1317240da9c94292cf6ef2e25 2023-10-28 00:00:28,715 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink dcf9afa91e314f178c59ced5a25fa944 of snapshot=snaptb-1698451213243 to region=795595fcd155993336d014aba73423ea table=testCompactionWithHFileLink 2023-10-28 00:00:28,719 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d39ca93432034ad29297bc42a5b0ea49 of snapshot=snaptb-1698451213243 to region=6d789d75232206c2589efabacffe9d96 table=testCompactionWithHFileLink 2023-10-28 00:00:28,719 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 4765103779e64954835a94c7a00e92a0 of snapshot=snaptb-1698451213243 to region=4904326d2e05286728bb6d35b919e5bf table=testCompactionWithHFileLink 2023-10-28 00:00:28,719 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 30493979777049ce8ebd4cf3bc0cd5d5 of snapshot=snaptb-1698451213243 to region=795595fcd155993336d014aba73423ea table=testCompactionWithHFileLink 2023-10-28 00:00:28,729 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 486a9ea7cf6e48928fb3a88650c0f7e6 of snapshot=snaptb-1698451213243 to region=6d789d75232206c2589efabacffe9d96 table=testCompactionWithHFileLink 2023-10-28 00:00:28,729 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 2cb1c76d629141179060550b5919816b of snapshot=snaptb-1698451213243 to region=795595fcd155993336d014aba73423ea table=testCompactionWithHFileLink 2023-10-28 00:00:28,729 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9a188e707c0a4959b84ee66b9e2b7085 of snapshot=snaptb-1698451213243 to region=4904326d2e05286728bb6d35b919e5bf table=testCompactionWithHFileLink 2023-10-28 00:00:28,735 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 1f3a75cc77e74b86b83067b7864d8a22 of snapshot=snaptb-1698451213243 to region=4904326d2e05286728bb6d35b919e5bf table=testCompactionWithHFileLink 2023-10-28 00:00:28,735 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 1ac14e9e9ba54d8f8eae2adbb5a965e9 of snapshot=snaptb-1698451213243 to region=795595fcd155993336d014aba73423ea table=testCompactionWithHFileLink 2023-10-28 00:00:28,735 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c61d558f355843b0aec0e0a9e2c97536 of snapshot=snaptb-1698451213243 to region=6d789d75232206c2589efabacffe9d96 table=testCompactionWithHFileLink 2023-10-28 00:00:28,744 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/8c339dc3e6cd46efb3e49040d9c4414c to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/8c339dc3e6cd46efb3e49040d9c4414c 2023-10-28 00:00:28,744 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/9769baa6b5394d9cbea76a76732c0d3d to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/9769baa6b5394d9cbea76a76732c0d3d 2023-10-28 00:00:28,744 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f854e3377f064dfc805f6e6239b2242e of snapshot=snaptb-1698451213243 to region=6d789d75232206c2589efabacffe9d96 table=testCompactionWithHFileLink 2023-10-28 00:00:28,744 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 2bfcfdf09d734e5dac83a9d255bd7680 of snapshot=snaptb-1698451213243 to region=795595fcd155993336d014aba73423ea table=testCompactionWithHFileLink 2023-10-28 00:00:28,744 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/479a91f53d9d43dcbdf8d8fa0df120d0 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/479a91f53d9d43dcbdf8d8fa0df120d0 2023-10-28 00:00:28,744 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 974f6211ca1740f2a34fe077aa91bdf2 of snapshot=snaptb-1698451213243 to region=4904326d2e05286728bb6d35b919e5bf table=testCompactionWithHFileLink 2023-10-28 00:00:28,752 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 2083b1a0e27a4323afa5cc383672554b of snapshot=snaptb-1698451213243 to region=795595fcd155993336d014aba73423ea table=testCompactionWithHFileLink 2023-10-28 00:00:28,752 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 2b1ba95eb4554512b2aa7f4f3bf684a7 of snapshot=snaptb-1698451213243 to region=4904326d2e05286728bb6d35b919e5bf table=testCompactionWithHFileLink 2023-10-28 00:00:28,752 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 69967cf4dac4418fb3e0be94fc433f04 of snapshot=snaptb-1698451213243 to region=6d789d75232206c2589efabacffe9d96 table=testCompactionWithHFileLink 2023-10-28 00:00:28,756 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 4a1c8f8ec1b744a293ebf977fdd7f04d of snapshot=snaptb-1698451213243 to region=795595fcd155993336d014aba73423ea table=testCompactionWithHFileLink 2023-10-28 00:00:28,756 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 01571f5665324edb985a32da4c4341a4 of snapshot=snaptb-1698451213243 to region=4904326d2e05286728bb6d35b919e5bf table=testCompactionWithHFileLink 2023-10-28 00:00:28,756 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 2948c86034f5417187a0810c68ad2363 of snapshot=snaptb-1698451213243 to region=6d789d75232206c2589efabacffe9d96 table=testCompactionWithHFileLink 2023-10-28 00:00:28,758 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 1fc20602194041dda4e593b5943c1c21 of snapshot=snaptb-1698451213243 to region=795595fcd155993336d014aba73423ea table=testCompactionWithHFileLink 2023-10-28 00:00:28,758 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink af19a53e844a4102a55235df865fc030 of snapshot=snaptb-1698451213243 to region=4904326d2e05286728bb6d35b919e5bf table=testCompactionWithHFileLink 2023-10-28 00:00:28,759 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 2f39466607184138b07dbe55159c1cf1 of snapshot=snaptb-1698451213243 to region=6d789d75232206c2589efabacffe9d96 table=testCompactionWithHFileLink 2023-10-28 00:00:28,762 INFO [PEWorker-1] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1698451213243" table: "testCompactionWithHFileLink" creation_time: 1698451213255 type: FLUSH version: 2 ttl: 0 2023-10-28 00:00:28,763 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1698451228763"}]},"ts":"9223372036854775807"} 2023-10-28 00:00:28,763 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1698451228763"}]},"ts":"9223372036854775807"} 2023-10-28 00:00:28,763 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1698451228763"}]},"ts":"9223372036854775807"} 2023-10-28 00:00:28,764 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-10-28 00:00:28,764 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 4904326d2e05286728bb6d35b919e5bf, NAME => 'testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => 6d789d75232206c2589efabacffe9d96, NAME => 'testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => 795595fcd155993336d014aba73423ea, NAME => 'testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea.', STARTKEY => '', ENDKEY => '1'}] 2023-10-28 00:00:28,764 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698451228764"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451228764"}]},"ts":"1698451228764"} 2023-10-28 00:00:28,764 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1698451228764"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451228764"}]},"ts":"1698451228764"} 2023-10-28 00:00:28,765 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698451228764"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451228764"}]},"ts":"1698451228764"} 2023-10-28 00:00:28,766 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-10-28 00:00:28,766 INFO [PEWorker-1] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-10-28 00:00:28,766 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 4904326d2e05286728bb6d35b919e5bf, NAME => 'testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => 6d789d75232206c2589efabacffe9d96, NAME => 'testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => 795595fcd155993336d014aba73423ea, NAME => 'testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea.', STARTKEY => '', ENDKEY => '1'}] 2023-10-28 00:00:28,766 INFO [PEWorker-1] procedure.RestoreSnapshotProcedure(457): Restore snapshot={ ss=snaptb-1698451213243 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-10-28 00:00:28,768 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1698451213243" table: "testCompactionWithHFileLink" creation_time: 1698451213255 type: FLUSH version: 2 ttl: 0 ) in 552 msec 2023-10-28 00:00:28,831 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-10-28 00:00:28,831 INFO [Listener at localhost/40295] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-10-28 00:00:28,832 INFO [Listener at localhost/40295] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1698451213243-1698451226038 2023-10-28 00:00:28,836 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(751): Client=jenkins//188.40.62.62 delete name: "hbase-failsafe-snaptb-1698451213243-1698451226038" 2023-10-28 00:00:28,838 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] snapshot.SnapshotManager(335): Deleting snapshot: hbase-failsafe-snaptb-1698451213243-1698451226038 2023-10-28 00:00:28,841 INFO [Listener at localhost/40295] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-10-28 00:00:28,845 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.HMaster$10(2385): Client=jenkins//188.40.62.62 enable testCompactionWithHFileLink 2023-10-28 00:00:28,852 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure2.ProcedureExecutor(1028): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-10-28 00:00:28,854 INFO [PEWorker-3] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-10-28 00:00:28,855 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451228855"}]},"ts":"1698451228855"} 2023-10-28 00:00:28,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=97 2023-10-28 00:00:28,856 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-10-28 00:00:28,925 DEBUG [PEWorker-3] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-10-28 00:00:28,925 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=795595fcd155993336d014aba73423ea, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6d789d75232206c2589efabacffe9d96, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=4904326d2e05286728bb6d35b919e5bf, ASSIGN}] 2023-10-28 00:00:28,927 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=795595fcd155993336d014aba73423ea, ASSIGN 2023-10-28 00:00:28,927 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=4904326d2e05286728bb6d35b919e5bf, ASSIGN 2023-10-28 00:00:28,927 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6d789d75232206c2589efabacffe9d96, ASSIGN 2023-10-28 00:00:28,927 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=795595fcd155993336d014aba73423ea, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-10-28 00:00:28,927 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=4904326d2e05286728bb6d35b919e5bf, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-10-28 00:00:28,927 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6d789d75232206c2589efabacffe9d96, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-10-28 00:00:28,956 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=97 2023-10-28 00:00:29,079 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=6d789d75232206c2589efabacffe9d96, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:29,079 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=4904326d2e05286728bb6d35b919e5bf, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:29,079 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1698451229079"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451229079"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451229079"}]},"ts":"1698451229079"} 2023-10-28 00:00:29,079 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=795595fcd155993336d014aba73423ea, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:29,079 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698451229079"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451229079"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451229079"}]},"ts":"1698451229079"} 2023-10-28 00:00:29,080 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698451229079"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451229079"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451229079"}]},"ts":"1698451229079"} 2023-10-28 00:00:29,081 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=101, ppid=99, state=RUNNABLE; OpenRegionProcedure 6d789d75232206c2589efabacffe9d96, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-28 00:00:29,082 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=102, ppid=100, state=RUNNABLE; OpenRegionProcedure 4904326d2e05286728bb6d35b919e5bf, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-28 00:00:29,083 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=103, ppid=98, state=RUNNABLE; OpenRegionProcedure 795595fcd155993336d014aba73423ea, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-28 00:00:29,158 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=97 2023-10-28 00:00:29,233 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:29,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:29,240 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6d789d75232206c2589efabacffe9d96, NAME => 'testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96.', STARTKEY => '1', ENDKEY => '2'} 2023-10-28 00:00:29,240 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 6d789d75232206c2589efabacffe9d96 2023-10-28 00:00:29,240 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-28 00:00:29,241 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for 6d789d75232206c2589efabacffe9d96 2023-10-28 00:00:29,241 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for 6d789d75232206c2589efabacffe9d96 2023-10-28 00:00:29,244 INFO [StoreOpener-6d789d75232206c2589efabacffe9d96-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 6d789d75232206c2589efabacffe9d96 2023-10-28 00:00:29,245 DEBUG [StoreOpener-6d789d75232206c2589efabacffe9d96-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1 2023-10-28 00:00:29,246 DEBUG [StoreOpener-6d789d75232206c2589efabacffe9d96-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1 2023-10-28 00:00:29,246 INFO [StoreOpener-6d789d75232206c2589efabacffe9d96-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6d789d75232206c2589efabacffe9d96 columnFamilyName family1 2023-10-28 00:00:29,256 DEBUG [StoreOpener-6d789d75232206c2589efabacffe9d96-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-486a9ea7cf6e48928fb3a88650c0f7e6 2023-10-28 00:00:29,263 DEBUG [StoreOpener-6d789d75232206c2589efabacffe9d96-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-7236d93111f746b6b44c723fc9e64af8 2023-10-28 00:00:29,269 DEBUG [StoreOpener-6d789d75232206c2589efabacffe9d96-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-c61d558f355843b0aec0e0a9e2c97536 2023-10-28 00:00:29,277 DEBUG [StoreOpener-6d789d75232206c2589efabacffe9d96-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-d39ca93432034ad29297bc42a5b0ea49 2023-10-28 00:00:29,277 INFO [StoreOpener-6d789d75232206c2589efabacffe9d96-1] regionserver.HStore(310): Store=6d789d75232206c2589efabacffe9d96/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-28 00:00:29,278 INFO [StoreOpener-6d789d75232206c2589efabacffe9d96-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 6d789d75232206c2589efabacffe9d96 2023-10-28 00:00:29,279 DEBUG [StoreOpener-6d789d75232206c2589efabacffe9d96-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2 2023-10-28 00:00:29,279 DEBUG [StoreOpener-6d789d75232206c2589efabacffe9d96-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2 2023-10-28 00:00:29,279 INFO [StoreOpener-6d789d75232206c2589efabacffe9d96-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6d789d75232206c2589efabacffe9d96 columnFamilyName family2 2023-10-28 00:00:29,288 DEBUG [StoreOpener-6d789d75232206c2589efabacffe9d96-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-2948c86034f5417187a0810c68ad2363 2023-10-28 00:00:29,294 DEBUG [StoreOpener-6d789d75232206c2589efabacffe9d96-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-2f39466607184138b07dbe55159c1cf1 2023-10-28 00:00:29,301 DEBUG [StoreOpener-6d789d75232206c2589efabacffe9d96-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-69967cf4dac4418fb3e0be94fc433f04 2023-10-28 00:00:29,309 DEBUG [StoreOpener-6d789d75232206c2589efabacffe9d96-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-f854e3377f064dfc805f6e6239b2242e 2023-10-28 00:00:29,309 INFO [StoreOpener-6d789d75232206c2589efabacffe9d96-1] regionserver.HStore(310): Store=6d789d75232206c2589efabacffe9d96/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-28 00:00:29,310 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96 2023-10-28 00:00:29,311 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96 2023-10-28 00:00:29,313 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-10-28 00:00:29,314 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for 6d789d75232206c2589efabacffe9d96 2023-10-28 00:00:29,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened 6d789d75232206c2589efabacffe9d96; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71143102, jitterRate=0.06011483073234558}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-10-28 00:00:29,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for 6d789d75232206c2589efabacffe9d96: 2023-10-28 00:00:29,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96., pid=101, masterSystemTime=1698451229233 2023-10-28 00:00:29,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-28 00:00:29,317 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-10-28 00:00:29,317 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-10-28 00:00:29,317 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-10-28 00:00:29,319 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:29,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:29,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:29,319 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4904326d2e05286728bb6d35b919e5bf, NAME => 'testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf.', STARTKEY => '2', ENDKEY => ''} 2023-10-28 00:00:29,319 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=6d789d75232206c2589efabacffe9d96, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:29,319 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1698451229319"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451229319"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451229319"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451229319"}]},"ts":"1698451229319"} 2023-10-28 00:00:29,319 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 4904326d2e05286728bb6d35b919e5bf 2023-10-28 00:00:29,320 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-28 00:00:29,320 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for 4904326d2e05286728bb6d35b919e5bf 2023-10-28 00:00:29,320 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for 4904326d2e05286728bb6d35b919e5bf 2023-10-28 00:00:29,320 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-10-28 00:00:29,320 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1912): 6d789d75232206c2589efabacffe9d96/family1 is initiating minor compaction (all files) 2023-10-28 00:00:29,320 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 6d789d75232206c2589efabacffe9d96/family1 in testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:29,320 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-7236d93111f746b6b44c723fc9e64af8, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-d39ca93432034ad29297bc42a5b0ea49, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-486a9ea7cf6e48928fb3a88650c0f7e6, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-c61d558f355843b0aec0e0a9e2c97536] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/.tmp, totalSize=21.6 K 2023-10-28 00:00:29,321 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-10-28 00:00:29,321 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-7236d93111f746b6b44c723fc9e64af8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698451205462 2023-10-28 00:00:29,321 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.HStore(1912): 6d789d75232206c2589efabacffe9d96/family2 is initiating minor compaction (all files) 2023-10-28 00:00:29,321 INFO [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 6d789d75232206c2589efabacffe9d96/family2 in testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:00:29,321 INFO [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-f854e3377f064dfc805f6e6239b2242e, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-69967cf4dac4418fb3e0be94fc433f04, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-2948c86034f5417187a0810c68ad2363, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-2f39466607184138b07dbe55159c1cf1] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/.tmp, totalSize=20.4 K 2023-10-28 00:00:29,322 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-d39ca93432034ad29297bc42a5b0ea49, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698451207430 2023-10-28 00:00:29,322 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-f854e3377f064dfc805f6e6239b2242e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698451205462 2023-10-28 00:00:29,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=101, resume processing ppid=99 2023-10-28 00:00:29,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=101, ppid=99, state=SUCCESS; OpenRegionProcedure 6d789d75232206c2589efabacffe9d96, server=jenkins-hbase16.apache.org,43469,1698451096272 in 240 msec 2023-10-28 00:00:29,323 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6d789d75232206c2589efabacffe9d96, ASSIGN in 397 msec 2023-10-28 00:00:29,323 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-486a9ea7cf6e48928fb3a88650c0f7e6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698451209411 2023-10-28 00:00:29,323 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-69967cf4dac4418fb3e0be94fc433f04, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698451207430 2023-10-28 00:00:29,324 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-c61d558f355843b0aec0e0a9e2c97536, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698451211339 2023-10-28 00:00:29,324 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-2948c86034f5417187a0810c68ad2363, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698451209411 2023-10-28 00:00:29,324 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-2f39466607184138b07dbe55159c1cf1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698451211339 2023-10-28 00:00:29,332 INFO [StoreOpener-4904326d2e05286728bb6d35b919e5bf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 4904326d2e05286728bb6d35b919e5bf 2023-10-28 00:00:29,333 DEBUG [StoreOpener-4904326d2e05286728bb6d35b919e5bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1 2023-10-28 00:00:29,333 DEBUG [StoreOpener-4904326d2e05286728bb6d35b919e5bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1 2023-10-28 00:00:29,333 INFO [StoreOpener-4904326d2e05286728bb6d35b919e5bf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4904326d2e05286728bb6d35b919e5bf columnFamilyName family1 2023-10-28 00:00:29,349 DEBUG [StoreOpener-4904326d2e05286728bb6d35b919e5bf-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-1f3a75cc77e74b86b83067b7864d8a22 2023-10-28 00:00:29,358 DEBUG [StoreOpener-4904326d2e05286728bb6d35b919e5bf-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-4765103779e64954835a94c7a00e92a0 2023-10-28 00:00:29,368 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6d789d75232206c2589efabacffe9d96#family1#compaction#114 average throughput is 0.26 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 57.69 MB/second 2023-10-28 00:00:29,370 DEBUG [StoreOpener-4904326d2e05286728bb6d35b919e5bf-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-9a188e707c0a4959b84ee66b9e2b7085 2023-10-28 00:00:29,371 INFO [RS:0;jenkins-hbase16:43469-longCompactions-0] throttle.PressureAwareThroughputController(145): 6d789d75232206c2589efabacffe9d96#family2#compaction#115 average throughput is 0.29 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 57.69 MB/second 2023-10-28 00:00:29,383 DEBUG [StoreOpener-4904326d2e05286728bb6d35b919e5bf-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-9e88a17ec01944c9a13e6ecf7608bf19 2023-10-28 00:00:29,384 INFO [StoreOpener-4904326d2e05286728bb6d35b919e5bf-1] regionserver.HStore(310): Store=4904326d2e05286728bb6d35b919e5bf/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-28 00:00:29,384 INFO [StoreOpener-4904326d2e05286728bb6d35b919e5bf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 4904326d2e05286728bb6d35b919e5bf 2023-10-28 00:00:29,385 DEBUG [StoreOpener-4904326d2e05286728bb6d35b919e5bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2 2023-10-28 00:00:29,385 DEBUG [StoreOpener-4904326d2e05286728bb6d35b919e5bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2 2023-10-28 00:00:29,385 INFO [StoreOpener-4904326d2e05286728bb6d35b919e5bf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4904326d2e05286728bb6d35b919e5bf columnFamilyName family2 2023-10-28 00:00:29,400 DEBUG [StoreOpener-4904326d2e05286728bb6d35b919e5bf-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-01571f5665324edb985a32da4c4341a4 2023-10-28 00:00:29,404 DEBUG [StoreOpener-4904326d2e05286728bb6d35b919e5bf-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-2b1ba95eb4554512b2aa7f4f3bf684a7 2023-10-28 00:00:29,408 DEBUG [StoreOpener-4904326d2e05286728bb6d35b919e5bf-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-974f6211ca1740f2a34fe077aa91bdf2 2023-10-28 00:00:29,413 DEBUG [StoreOpener-4904326d2e05286728bb6d35b919e5bf-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-af19a53e844a4102a55235df865fc030 2023-10-28 00:00:29,413 INFO [StoreOpener-4904326d2e05286728bb6d35b919e5bf-1] regionserver.HStore(310): Store=4904326d2e05286728bb6d35b919e5bf/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-28 00:00:29,414 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf 2023-10-28 00:00:29,414 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf 2023-10-28 00:00:29,416 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-10-28 00:00:29,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for 4904326d2e05286728bb6d35b919e5bf 2023-10-28 00:00:29,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened 4904326d2e05286728bb6d35b919e5bf; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63956704, jitterRate=-0.04697084426879883}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-10-28 00:00:29,418 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for 4904326d2e05286728bb6d35b919e5bf: 2023-10-28 00:00:29,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf., pid=102, masterSystemTime=1698451229233 2023-10-28 00:00:29,419 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-10-28 00:00:29,419 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-10-28 00:00:29,420 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:29,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:29,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:29,420 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 795595fcd155993336d014aba73423ea, NAME => 'testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea.', STARTKEY => '', ENDKEY => '1'} 2023-10-28 00:00:29,420 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=4904326d2e05286728bb6d35b919e5bf, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:29,421 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698451229420"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451229420"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451229420"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451229420"}]},"ts":"1698451229420"} 2023-10-28 00:00:29,421 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 795595fcd155993336d014aba73423ea 2023-10-28 00:00:29,421 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-28 00:00:29,421 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for 795595fcd155993336d014aba73423ea 2023-10-28 00:00:29,421 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for 795595fcd155993336d014aba73423ea 2023-10-28 00:00:29,422 INFO [StoreOpener-795595fcd155993336d014aba73423ea-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 795595fcd155993336d014aba73423ea 2023-10-28 00:00:29,423 DEBUG [StoreOpener-795595fcd155993336d014aba73423ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1 2023-10-28 00:00:29,423 DEBUG [StoreOpener-795595fcd155993336d014aba73423ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1 2023-10-28 00:00:29,423 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=102, resume processing ppid=100 2023-10-28 00:00:29,423 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=102, ppid=100, state=SUCCESS; OpenRegionProcedure 4904326d2e05286728bb6d35b919e5bf, server=jenkins-hbase16.apache.org,43469,1698451096272 in 340 msec 2023-10-28 00:00:29,424 INFO [StoreOpener-795595fcd155993336d014aba73423ea-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 795595fcd155993336d014aba73423ea columnFamilyName family1 2023-10-28 00:00:29,424 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=4904326d2e05286728bb6d35b919e5bf, ASSIGN in 498 msec 2023-10-28 00:00:29,430 DEBUG [StoreOpener-795595fcd155993336d014aba73423ea-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-1ac14e9e9ba54d8f8eae2adbb5a965e9 2023-10-28 00:00:29,437 DEBUG [StoreOpener-795595fcd155993336d014aba73423ea-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-2cb1c76d629141179060550b5919816b 2023-10-28 00:00:29,441 DEBUG [StoreOpener-795595fcd155993336d014aba73423ea-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-30493979777049ce8ebd4cf3bc0cd5d5 2023-10-28 00:00:29,446 DEBUG [StoreOpener-795595fcd155993336d014aba73423ea-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-dcf9afa91e314f178c59ced5a25fa944 2023-10-28 00:00:29,446 INFO [StoreOpener-795595fcd155993336d014aba73423ea-1] regionserver.HStore(310): Store=795595fcd155993336d014aba73423ea/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-28 00:00:29,446 INFO [StoreOpener-795595fcd155993336d014aba73423ea-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 795595fcd155993336d014aba73423ea 2023-10-28 00:00:29,447 DEBUG [StoreOpener-795595fcd155993336d014aba73423ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2 2023-10-28 00:00:29,447 DEBUG [StoreOpener-795595fcd155993336d014aba73423ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2 2023-10-28 00:00:29,447 INFO [StoreOpener-795595fcd155993336d014aba73423ea-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 795595fcd155993336d014aba73423ea columnFamilyName family2 2023-10-28 00:00:29,455 DEBUG [StoreOpener-795595fcd155993336d014aba73423ea-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-1fc20602194041dda4e593b5943c1c21 2023-10-28 00:00:29,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=97 2023-10-28 00:00:29,461 DEBUG [StoreOpener-795595fcd155993336d014aba73423ea-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-2083b1a0e27a4323afa5cc383672554b 2023-10-28 00:00:29,467 DEBUG [StoreOpener-795595fcd155993336d014aba73423ea-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-2bfcfdf09d734e5dac83a9d255bd7680 2023-10-28 00:00:29,473 DEBUG [StoreOpener-795595fcd155993336d014aba73423ea-1] regionserver.HStore(539): loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-4a1c8f8ec1b744a293ebf977fdd7f04d 2023-10-28 00:00:29,473 INFO [StoreOpener-795595fcd155993336d014aba73423ea-1] regionserver.HStore(310): Store=795595fcd155993336d014aba73423ea/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-28 00:00:29,474 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea 2023-10-28 00:00:29,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea 2023-10-28 00:00:29,476 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-10-28 00:00:29,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for 795595fcd155993336d014aba73423ea 2023-10-28 00:00:29,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened 795595fcd155993336d014aba73423ea; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60297698, jitterRate=-0.10149428248405457}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-10-28 00:00:29,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for 795595fcd155993336d014aba73423ea: 2023-10-28 00:00:29,478 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea., pid=103, masterSystemTime=1698451229233 2023-10-28 00:00:29,478 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-10-28 00:00:29,478 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-10-28 00:00:29,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:29,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:29,480 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=795595fcd155993336d014aba73423ea, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:29,480 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698451229480"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451229480"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451229480"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451229480"}]},"ts":"1698451229480"} 2023-10-28 00:00:29,483 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=103, resume processing ppid=98 2023-10-28 00:00:29,483 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=103, ppid=98, state=SUCCESS; OpenRegionProcedure 795595fcd155993336d014aba73423ea, server=jenkins-hbase16.apache.org,43469,1698451096272 in 399 msec 2023-10-28 00:00:29,484 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=98, resume processing ppid=97 2023-10-28 00:00:29,484 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=795595fcd155993336d014aba73423ea, ASSIGN in 558 msec 2023-10-28 00:00:29,485 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451229485"}]},"ts":"1698451229485"} 2023-10-28 00:00:29,486 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-10-28 00:00:29,499 INFO [PEWorker-3] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-10-28 00:00:29,501 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 653 msec 2023-10-28 00:00:29,814 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/.tmp/family2/0c4664fa4cd4440680dcdad2650995b9 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/0c4664fa4cd4440680dcdad2650995b9 2023-10-28 00:00:29,821 INFO [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 6d789d75232206c2589efabacffe9d96/family2 of 6d789d75232206c2589efabacffe9d96 into 0c4664fa4cd4440680dcdad2650995b9(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-10-28 00:00:29,821 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6d789d75232206c2589efabacffe9d96: 2023-10-28 00:00:29,821 INFO [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96., storeName=6d789d75232206c2589efabacffe9d96/family2, priority=12, startTime=1698451229317; duration=0sec 2023-10-28 00:00:29,821 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-10-28 00:00:29,822 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-10-28 00:00:29,826 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-10-28 00:00:29,826 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.HStore(1912): 4904326d2e05286728bb6d35b919e5bf/family1 is initiating minor compaction (all files) 2023-10-28 00:00:29,826 INFO [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 4904326d2e05286728bb6d35b919e5bf/family1 in testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:29,827 INFO [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-9e88a17ec01944c9a13e6ecf7608bf19, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-4765103779e64954835a94c7a00e92a0, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-9a188e707c0a4959b84ee66b9e2b7085, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-1f3a75cc77e74b86b83067b7864d8a22] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/.tmp, totalSize=21.6 K 2023-10-28 00:00:29,827 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-9e88a17ec01944c9a13e6ecf7608bf19, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698451205463 2023-10-28 00:00:29,828 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-4765103779e64954835a94c7a00e92a0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698451207431 2023-10-28 00:00:29,829 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-9a188e707c0a4959b84ee66b9e2b7085, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698451209412 2023-10-28 00:00:29,831 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-1f3a75cc77e74b86b83067b7864d8a22, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698451211340 2023-10-28 00:00:29,835 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-f854e3377f064dfc805f6e6239b2242e, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-69967cf4dac4418fb3e0be94fc433f04, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-2948c86034f5417187a0810c68ad2363, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-2f39466607184138b07dbe55159c1cf1] to archive 2023-10-28 00:00:29,836 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:29,838 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-f854e3377f064dfc805f6e6239b2242e to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-f854e3377f064dfc805f6e6239b2242e 2023-10-28 00:00:29,840 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-69967cf4dac4418fb3e0be94fc433f04 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-69967cf4dac4418fb3e0be94fc433f04 2023-10-28 00:00:29,842 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-2948c86034f5417187a0810c68ad2363 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-2948c86034f5417187a0810c68ad2363 2023-10-28 00:00:29,844 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-2f39466607184138b07dbe55159c1cf1 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family2/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-2f39466607184138b07dbe55159c1cf1 2023-10-28 00:00:29,855 INFO [RS:0;jenkins-hbase16:43469-longCompactions-0] throttle.PressureAwareThroughputController(145): 4904326d2e05286728bb6d35b919e5bf#family1#compaction#116 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 57.69 MB/second 2023-10-28 00:00:29,961 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=97 2023-10-28 00:00:29,962 INFO [Listener at localhost/40295] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-10-28 00:00:29,983 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-10-28 00:00:29,991 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-10-28 00:00:30,000 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-10-28 00:00:30,000 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-10-28 00:00:30,005 INFO [Listener at localhost/40295] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-10-28 00:00:30,019 INFO [Listener at localhost/40295] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-10-28 00:00:30,019 INFO [Listener at localhost/40295] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-10-28 00:00:30,020 DEBUG [Listener at localhost/40295] zookeeper.ReadOnlyZKClient(139): Connect 0x22171f55 to 127.0.0.1:57369 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-10-28 00:00:30,058 DEBUG [Listener at localhost/40295] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6244325a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-28 00:00:30,058 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20231028 for table='testCompactionWithHFileLink' and column='family1' 2023-10-28 00:00:30,059 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20231028 for table='testCompactionWithHFileLink' and column='family1' 2023-10-28 00:00:30,058 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20231028 for table='testCompactionWithHFileLink' and column='family1' 2023-10-28 00:00:30,613 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/.tmp/family1/cdfca916d2ad4218ae424f9b75669f4e as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/cdfca916d2ad4218ae424f9b75669f4e 2023-10-28 00:00:30,619 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 6d789d75232206c2589efabacffe9d96/family1 of 6d789d75232206c2589efabacffe9d96 into cdfca916d2ad4218ae424f9b75669f4e(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-10-28 00:00:30,619 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6d789d75232206c2589efabacffe9d96: 2023-10-28 00:00:30,619 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96., storeName=6d789d75232206c2589efabacffe9d96/family1, priority=12, startTime=1698451229316; duration=1sec 2023-10-28 00:00:30,619 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-10-28 00:00:30,620 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-10-28 00:00:30,624 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-10-28 00:00:30,624 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1912): 795595fcd155993336d014aba73423ea/family2 is initiating minor compaction (all files) 2023-10-28 00:00:30,624 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 795595fcd155993336d014aba73423ea/family2 in testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:30,624 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-2bfcfdf09d734e5dac83a9d255bd7680, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-2083b1a0e27a4323afa5cc383672554b, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-4a1c8f8ec1b744a293ebf977fdd7f04d, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-1fc20602194041dda4e593b5943c1c21] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/.tmp, totalSize=20.4 K 2023-10-28 00:00:30,625 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-2bfcfdf09d734e5dac83a9d255bd7680, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698451205461 2023-10-28 00:00:30,626 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-2083b1a0e27a4323afa5cc383672554b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698451207430 2023-10-28 00:00:30,627 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-4a1c8f8ec1b744a293ebf977fdd7f04d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698451209411 2023-10-28 00:00:30,628 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-1fc20602194041dda4e593b5943c1c21, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698451211339 2023-10-28 00:00:30,643 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-7236d93111f746b6b44c723fc9e64af8, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-d39ca93432034ad29297bc42a5b0ea49, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-486a9ea7cf6e48928fb3a88650c0f7e6, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-c61d558f355843b0aec0e0a9e2c97536] to archive 2023-10-28 00:00:30,644 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:30,647 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-7236d93111f746b6b44c723fc9e64af8 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-7236d93111f746b6b44c723fc9e64af8 2023-10-28 00:00:30,649 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-d39ca93432034ad29297bc42a5b0ea49 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-d39ca93432034ad29297bc42a5b0ea49 2023-10-28 00:00:30,651 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-486a9ea7cf6e48928fb3a88650c0f7e6 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-486a9ea7cf6e48928fb3a88650c0f7e6 2023-10-28 00:00:30,652 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-c61d558f355843b0aec0e0a9e2c97536 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/testCompactionWithHFileLink=6d789d75232206c2589efabacffe9d96-c61d558f355843b0aec0e0a9e2c97536 2023-10-28 00:00:30,653 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] throttle.PressureAwareThroughputController(145): 795595fcd155993336d014aba73423ea#family2#compaction#117 average throughput is 0.39 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 57.69 MB/second 2023-10-28 00:00:30,965 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023102890feccc05de04ad2af740320a4028f2d to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023102890feccc05de04ad2af740320a4028f2d 2023-10-28 00:00:30,966 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202310286694adcb15f0492586074c32cf5aa049 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202310286694adcb15f0492586074c32cf5aa049 2023-10-28 00:00:30,966 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231028945e72956cff4f99ab0be1f1f3b63890 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231028945e72956cff4f99ab0be1f1f3b63890 2023-10-28 00:00:30,966 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231028 table='testCompactionWithHFileLink' column='family1' 2023-10-28 00:00:30,967 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231028 table='testCompactionWithHFileLink' column='family1' 2023-10-28 00:00:30,967 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231028 table='testCompactionWithHFileLink' column='family1' 2023-10-28 00:00:30,979 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-10-28 00:00:30,990 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:34940, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-10-28 00:00:30,996 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-10-28 00:00:30,999 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:44958, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-10-28 00:00:31,019 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231028/family1/eddc5f14262c45cc8c6ec2e29bdf2cee first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-10-28 00:00:31,021 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231028/family1/eddc5f14262c45cc8c6ec2e29bdf2cee}] 2023-10-28 00:00:31,024 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231028/family1/caa19debf77f4ca3a5712835a8e0d0c4 first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-10-28 00:00:31,025 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231028/family1/1a724404bb3f4bd099989980385deb01 first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-10-28 00:00:31,027 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=32 for row with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231028/family1/caa19debf77f4ca3a5712835a8e0d0c4}] 2023-10-28 00:00:31,027 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231028/family1/1a724404bb3f4bd099989980385deb01}] 2023-10-28 00:00:31,030 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231028/family1/eddc5f14262c45cc8c6ec2e29bdf2cee for inclusion in 6d789d75232206c2589efabacffe9d96/family1 2023-10-28 00:00:31,030 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231028/family1/1a724404bb3f4bd099989980385deb01 for inclusion in 4904326d2e05286728bb6d35b919e5bf/family1 2023-10-28 00:00:31,030 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231028/family1/caa19debf77f4ca3a5712835a8e0d0c4 for inclusion in 795595fcd155993336d014aba73423ea/family1 2023-10-28 00:00:31,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-10-28 00:00:31,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(777): Region bounds: first= last=1 2023-10-28 00:00:31,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-10-28 00:00:31,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(777): Region bounds: first=1 last=2 2023-10-28 00:00:31,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for 795595fcd155993336d014aba73423ea: 2023-10-28 00:00:31,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for 6d789d75232206c2589efabacffe9d96: 2023-10-28 00:00:31,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-10-28 00:00:31,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43469] regionserver.HStore(777): Region bounds: first=2 last= 2023-10-28 00:00:31,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for 4904326d2e05286728bb6d35b919e5bf: 2023-10-28 00:00:31,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231028/family1/eddc5f14262c45cc8c6ec2e29bdf2cee to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__v3d6mk3t262vc48ngqvquujervfjdkjm6boir30mohopg8o0grnldaiuphcfs4o/family1/eddc5f14262c45cc8c6ec2e29bdf2cee 2023-10-28 00:00:31,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231028/family1/1a724404bb3f4bd099989980385deb01 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__jsmd534pgd9c3mle35lds8tvqnb1uir4nnfj71gk653mpq0omqpmennj80gp7ugo/family1/1a724404bb3f4bd099989980385deb01 2023-10-28 00:00:31,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231028/family1/caa19debf77f4ca3a5712835a8e0d0c4 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__n0chd2ctujklf982fi6cev1fqommfvpohq64hopln9mookpnsh3rkfh08q7e99ah/family1/caa19debf77f4ca3a5712835a8e0d0c4 2023-10-28 00:00:31,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__jsmd534pgd9c3mle35lds8tvqnb1uir4nnfj71gk653mpq0omqpmennj80gp7ugo/family1/1a724404bb3f4bd099989980385deb01 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/3439bdb4039e4a639f5899975c77d881_SeqId_34_ 2023-10-28 00:00:31,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__n0chd2ctujklf982fi6cev1fqommfvpohq64hopln9mookpnsh3rkfh08q7e99ah/family1/caa19debf77f4ca3a5712835a8e0d0c4 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/e4c492e9769a4c46a68b0ec6a5c0e6ad_SeqId_34_ 2023-10-28 00:00:31,035 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__jsmd534pgd9c3mle35lds8tvqnb1uir4nnfj71gk653mpq0omqpmennj80gp7ugo/family1/1a724404bb3f4bd099989980385deb01 into 4904326d2e05286728bb6d35b919e5bf/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/3439bdb4039e4a639f5899975c77d881_SeqId_34_ - updating store file list. 2023-10-28 00:00:31,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__v3d6mk3t262vc48ngqvquujervfjdkjm6boir30mohopg8o0grnldaiuphcfs4o/family1/eddc5f14262c45cc8c6ec2e29bdf2cee as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/6083214b5f6a44de9820dc571e271910_SeqId_36_ 2023-10-28 00:00:31,036 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__n0chd2ctujklf982fi6cev1fqommfvpohq64hopln9mookpnsh3rkfh08q7e99ah/family1/caa19debf77f4ca3a5712835a8e0d0c4 into 795595fcd155993336d014aba73423ea/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/e4c492e9769a4c46a68b0ec6a5c0e6ad_SeqId_34_ - updating store file list. 2023-10-28 00:00:31,036 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__v3d6mk3t262vc48ngqvquujervfjdkjm6boir30mohopg8o0grnldaiuphcfs4o/family1/eddc5f14262c45cc8c6ec2e29bdf2cee into 6d789d75232206c2589efabacffe9d96/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/6083214b5f6a44de9820dc571e271910_SeqId_36_ - updating store file list. 2023-10-28 00:00:31,039 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/3439bdb4039e4a639f5899975c77d881_SeqId_34_ into 4904326d2e05286728bb6d35b919e5bf/family1 2023-10-28 00:00:31,039 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__jsmd534pgd9c3mle35lds8tvqnb1uir4nnfj71gk653mpq0omqpmennj80gp7ugo/family1/1a724404bb3f4bd099989980385deb01 into 4904326d2e05286728bb6d35b919e5bf/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/3439bdb4039e4a639f5899975c77d881_SeqId_34_) 2023-10-28 00:00:31,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__jsmd534pgd9c3mle35lds8tvqnb1uir4nnfj71gk653mpq0omqpmennj80gp7ugo/family1/1a724404bb3f4bd099989980385deb01 2023-10-28 00:00:31,040 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/6083214b5f6a44de9820dc571e271910_SeqId_36_ into 6d789d75232206c2589efabacffe9d96/family1 2023-10-28 00:00:31,040 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/e4c492e9769a4c46a68b0ec6a5c0e6ad_SeqId_34_ into 795595fcd155993336d014aba73423ea/family1 2023-10-28 00:00:31,040 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__v3d6mk3t262vc48ngqvquujervfjdkjm6boir30mohopg8o0grnldaiuphcfs4o/family1/eddc5f14262c45cc8c6ec2e29bdf2cee into 6d789d75232206c2589efabacffe9d96/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/family1/6083214b5f6a44de9820dc571e271910_SeqId_36_) 2023-10-28 00:00:31,040 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__n0chd2ctujklf982fi6cev1fqommfvpohq64hopln9mookpnsh3rkfh08q7e99ah/family1/caa19debf77f4ca3a5712835a8e0d0c4 into 795595fcd155993336d014aba73423ea/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/e4c492e9769a4c46a68b0ec6a5c0e6ad_SeqId_34_) 2023-10-28 00:00:31,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__v3d6mk3t262vc48ngqvquujervfjdkjm6boir30mohopg8o0grnldaiuphcfs4o/family1/eddc5f14262c45cc8c6ec2e29bdf2cee 2023-10-28 00:00:31,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testCompactionWithHFileLink__n0chd2ctujklf982fi6cev1fqommfvpohq64hopln9mookpnsh3rkfh08q7e99ah/family1/caa19debf77f4ca3a5712835a8e0d0c4 2023-10-28 00:00:31,041 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-10-28 00:00:31,041 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231028 table='testCompactionWithHFileLink' column='family1' 2023-10-28 00:00:31,041 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:31,042 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231028 table='testCompactionWithHFileLink' column='family1' 2023-10-28 00:00:31,042 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202310283b7c07e40bf24d94844e40d254645d1f to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202310283b7c07e40bf24d94844e40d254645d1f 2023-10-28 00:00:31,043 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:31,043 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-10-28 00:00:31,043 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(1912): 795595fcd155993336d014aba73423ea/family1 is initiating minor compaction (all files) 2023-10-28 00:00:31,043 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@251cafa3; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-10-28 00:00:31,043 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023102895f8921a8d4d4d278d029ad418c53fc9 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023102895f8921a8d4d4d278d029ad418c53fc9 2023-10-28 00:00:31,044 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202310285de2655de28242baa74d1aa200b8e278 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202310285de2655de28242baa74d1aa200b8e278 2023-10-28 00:00:31,044 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202310289c1bbedd5cfa43539015c979c7f943db to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202310289c1bbedd5cfa43539015c979c7f943db 2023-10-28 00:00:31,045 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20231028b6ea31eec0664b918f7ee732a522247b to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20231028b6ea31eec0664b918f7ee732a522247b 2023-10-28 00:00:31,045 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231028 table='testCompactionWithHFileLink' column='family1' 2023-10-28 00:00:31,045 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202310289d878da8c1b44dcc97673584130fa660 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202310289d878da8c1b44dcc97673584130fa660 2023-10-28 00:00:31,045 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-10-28 00:00:31,045 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231028' 2023-10-28 00:00:31,045 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:31,046 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20231028 for table='testCompactionWithHFileLink' and column='family1' 2023-10-28 00:00:31,046 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20231028d89c12a20837409db30f53e7647e3333 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20231028d89c12a20837409db30f53e7647e3333 2023-10-28 00:00:31,046 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202310284c7a22bee8b540ad908824b1f36d2ef3 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202310284c7a22bee8b540ad908824b1f36d2ef3 2023-10-28 00:00:31,047 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20231028f1ccb541978d4e818c1e1cd3c99edddd to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20231028f1ccb541978d4e818c1e1cd3c99edddd 2023-10-28 00:00:31,047 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-10-28 00:00:31,047 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231028' 2023-10-28 00:00:31,047 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20231028 for table='testCompactionWithHFileLink' and column='family1' 2023-10-28 00:00:31,047 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20231028625ac864940d4814bc5b572d46243862 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20231028625ac864940d4814bc5b572d46243862 2023-10-28 00:00:31,048 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023102872405d973040401f805bd31e54b32525 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023102872405d973040401f805bd31e54b32525 2023-10-28 00:00:31,048 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202310289cf51282aea94d7d81e7ee5e8cb74330 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202310289cf51282aea94d7d81e7ee5e8cb74330 2023-10-28 00:00:31,048 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-10-28 00:00:31,048 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231028' 2023-10-28 00:00:31,049 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20231028 for table='testCompactionWithHFileLink' and column='family1' 2023-10-28 00:00:31,049 INFO [Listener at localhost/40295] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-10-28 00:00:31,051 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-10-28 00:00:31,054 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-10-28 00:00:31,056 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-10-28 00:00:31,057 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-10-28 00:00:31,063 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-10-28 00:00:31,064 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-10-28 00:00:31,077 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/.tmp/family1/5b9a495ea7e149b782cdcfc4394408e0 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/5b9a495ea7e149b782cdcfc4394408e0 2023-10-28 00:00:31,079 INFO [Listener at localhost/40295] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=280 (was 245) - Thread LEAK? -, OpenFileDescriptor=746 (was 606) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=82 (was 115), ProcessCount=168 (was 166) - ProcessCount LEAK? -, AvailableMemoryMB=5914 (was 5969) 2023-10-28 00:00:31,086 INFO [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 4904326d2e05286728bb6d35b919e5bf/family1 of 4904326d2e05286728bb6d35b919e5bf into 5b9a495ea7e149b782cdcfc4394408e0(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-10-28 00:00:31,086 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 4904326d2e05286728bb6d35b919e5bf: 2023-10-28 00:00:31,087 INFO [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf., storeName=4904326d2e05286728bb6d35b919e5bf/family1, priority=12, startTime=1698451229419; duration=1sec 2023-10-28 00:00:31,087 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-10-28 00:00:31,087 INFO [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 795595fcd155993336d014aba73423ea/family1 in testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:00:31,087 INFO [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-dcf9afa91e314f178c59ced5a25fa944, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-30493979777049ce8ebd4cf3bc0cd5d5, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-2cb1c76d629141179060550b5919816b, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-1ac14e9e9ba54d8f8eae2adbb5a965e9, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/e4c492e9769a4c46a68b0ec6a5c0e6ad_SeqId_34_] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/.tmp, totalSize=28.9 K 2023-10-28 00:00:31,088 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-dcf9afa91e314f178c59ced5a25fa944, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698451205461 2023-10-28 00:00:31,089 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-30493979777049ce8ebd4cf3bc0cd5d5, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698451207430 2023-10-28 00:00:31,089 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-2cb1c76d629141179060550b5919816b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698451209411 2023-10-28 00:00:31,090 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-1ac14e9e9ba54d8f8eae2adbb5a965e9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698451211339 2023-10-28 00:00:31,090 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] compactions.Compactor(207): Compacting e4c492e9769a4c46a68b0ec6a5c0e6ad_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1698451205461 2023-10-28 00:00:31,091 INFO [Listener at localhost/40295] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=279, OpenFileDescriptor=748, MaxFileDescriptor=60000, SystemLoadAverage=82, ProcessCount=168, AvailableMemoryMB=5914 2023-10-28 00:00:31,096 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] master.HMaster$15(3014): Client=jenkins//188.40.62.62 creating {NAME => 'ns'} 2023-10-28 00:00:31,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] procedure2.ProcedureExecutor(1028): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-10-28 00:00:31,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=104 2023-10-28 00:00:31,105 INFO [RS:0;jenkins-hbase16:43469-longCompactions-0] throttle.PressureAwareThroughputController(145): 795595fcd155993336d014aba73423ea#family1#compaction#118 average throughput is 2.33 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 57.69 MB/second 2023-10-28 00:00:31,133 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-10-28 00:00:31,134 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-9e88a17ec01944c9a13e6ecf7608bf19, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-4765103779e64954835a94c7a00e92a0, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-9a188e707c0a4959b84ee66b9e2b7085, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-1f3a75cc77e74b86b83067b7864d8a22] to archive 2023-10-28 00:00:31,135 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:31,137 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-9e88a17ec01944c9a13e6ecf7608bf19 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-9e88a17ec01944c9a13e6ecf7608bf19 2023-10-28 00:00:31,138 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-4765103779e64954835a94c7a00e92a0 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-4765103779e64954835a94c7a00e92a0 2023-10-28 00:00:31,140 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-9a188e707c0a4959b84ee66b9e2b7085 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-9a188e707c0a4959b84ee66b9e2b7085 2023-10-28 00:00:31,141 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-1f3a75cc77e74b86b83067b7864d8a22 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family1/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-1f3a75cc77e74b86b83067b7864d8a22 2023-10-28 00:00:31,143 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 44 msec 2023-10-28 00:00:31,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=104 2023-10-28 00:00:31,206 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] master.HMaster$4(2112): Client=jenkins//188.40.62.62 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-10-28 00:00:31,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] procedure2.ProcedureExecutor(1028): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-10-28 00:00:31,210 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-10-28 00:00:31,210 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] master.MasterRpcServices(700): Client=jenkins//188.40.62.62 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-10-28 00:00:31,211 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-10-28 00:00:31,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-10-28 00:00:31,215 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41 2023-10-28 00:00:31,215 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5 2023-10-28 00:00:31,215 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a 2023-10-28 00:00:31,215 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41 empty. 2023-10-28 00:00:31,216 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a empty. 2023-10-28 00:00:31,216 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5 empty. 2023-10-28 00:00:31,216 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41 2023-10-28 00:00:31,216 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a 2023-10-28 00:00:31,217 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5 2023-10-28 00:00:31,217 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-10-28 00:00:31,227 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-10-28 00:00:31,229 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => d4d6277130ca600bf099d77a935d3b41, NAME => 'ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp 2023-10-28 00:00:31,229 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7675): creating {ENCODED => e6aa79e78506420304e660bae0e1ad9a, NAME => 'ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp 2023-10-28 00:00:31,229 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7675): creating {ENCODED => f0ae0de72c21217ffb45facabb5a6cd5, NAME => 'ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp 2023-10-28 00:00:31,236 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-28 00:00:31,236 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1604): Closing e6aa79e78506420304e660bae0e1ad9a, disabling compactions & flushes 2023-10-28 00:00:31,236 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. 2023-10-28 00:00:31,236 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. 2023-10-28 00:00:31,237 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. after waiting 0 ms 2023-10-28 00:00:31,237 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. 2023-10-28 00:00:31,237 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. 2023-10-28 00:00:31,237 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1558): Region close journal for e6aa79e78506420304e660bae0e1ad9a: 2023-10-28 00:00:31,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-10-28 00:00:31,481 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-10-28 00:00:31,515 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-10-28 00:00:31,637 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-28 00:00:31,637 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-28 00:00:31,638 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1604): Closing f0ae0de72c21217ffb45facabb5a6cd5, disabling compactions & flushes 2023-10-28 00:00:31,638 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1604): Closing d4d6277130ca600bf099d77a935d3b41, disabling compactions & flushes 2023-10-28 00:00:31,638 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. 2023-10-28 00:00:31,638 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. 2023-10-28 00:00:31,638 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. after waiting 0 ms 2023-10-28 00:00:31,638 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. 2023-10-28 00:00:31,638 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. 2023-10-28 00:00:31,638 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. 2023-10-28 00:00:31,639 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. 2023-10-28 00:00:31,639 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. after waiting 0 ms 2023-10-28 00:00:31,639 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1558): Region close journal for d4d6277130ca600bf099d77a935d3b41: 2023-10-28 00:00:31,639 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. 2023-10-28 00:00:31,639 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. 2023-10-28 00:00:31,639 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1558): Region close journal for f0ae0de72c21217ffb45facabb5a6cd5: 2023-10-28 00:00:31,643 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-10-28 00:00:31,645 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1698451231645"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451231645"}]},"ts":"1698451231645"} 2023-10-28 00:00:31,645 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1698451231645"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451231645"}]},"ts":"1698451231645"} 2023-10-28 00:00:31,645 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1698451231645"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451231645"}]},"ts":"1698451231645"} 2023-10-28 00:00:31,648 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-10-28 00:00:31,649 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-10-28 00:00:31,649 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451231649"}]},"ts":"1698451231649"} 2023-10-28 00:00:31,650 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-10-28 00:00:31,691 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d4d6277130ca600bf099d77a935d3b41, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=f0ae0de72c21217ffb45facabb5a6cd5, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=e6aa79e78506420304e660bae0e1ad9a, ASSIGN}] 2023-10-28 00:00:31,693 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=e6aa79e78506420304e660bae0e1ad9a, ASSIGN 2023-10-28 00:00:31,693 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d4d6277130ca600bf099d77a935d3b41, ASSIGN 2023-10-28 00:00:31,693 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=f0ae0de72c21217ffb45facabb5a6cd5, ASSIGN 2023-10-28 00:00:31,694 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=e6aa79e78506420304e660bae0e1ad9a, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,43469,1698451096272; forceNewPlan=false, retain=false 2023-10-28 00:00:31,694 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=f0ae0de72c21217ffb45facabb5a6cd5, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,43469,1698451096272; forceNewPlan=false, retain=false 2023-10-28 00:00:31,694 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d4d6277130ca600bf099d77a935d3b41, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,43469,1698451096272; forceNewPlan=false, retain=false 2023-10-28 00:00:31,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-10-28 00:00:31,850 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=d4d6277130ca600bf099d77a935d3b41, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:31,850 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=f0ae0de72c21217ffb45facabb5a6cd5, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:31,850 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=e6aa79e78506420304e660bae0e1ad9a, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:31,851 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1698451231850"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451231850"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451231850"}]},"ts":"1698451231850"} 2023-10-28 00:00:31,850 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1698451231850"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451231850"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451231850"}]},"ts":"1698451231850"} 2023-10-28 00:00:31,851 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1698451231850"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451231850"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451231850"}]},"ts":"1698451231850"} 2023-10-28 00:00:31,854 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=109, ppid=107, state=RUNNABLE; OpenRegionProcedure f0ae0de72c21217ffb45facabb5a6cd5, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-28 00:00:31,856 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=110, ppid=106, state=RUNNABLE; OpenRegionProcedure d4d6277130ca600bf099d77a935d3b41, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-28 00:00:31,856 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=111, ppid=108, state=RUNNABLE; OpenRegionProcedure e6aa79e78506420304e660bae0e1ad9a, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-28 00:00:31,881 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/.tmp/family2/1e9c612aff8d4aaf8f4dd4b469858e81 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/1e9c612aff8d4aaf8f4dd4b469858e81 2023-10-28 00:00:31,888 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 795595fcd155993336d014aba73423ea/family2 of 795595fcd155993336d014aba73423ea into 1e9c612aff8d4aaf8f4dd4b469858e81(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-10-28 00:00:31,888 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 795595fcd155993336d014aba73423ea: 2023-10-28 00:00:31,888 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea., storeName=795595fcd155993336d014aba73423ea/family2, priority=12, startTime=1698451229478; duration=1sec 2023-10-28 00:00:31,888 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-10-28 00:00:31,888 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 5 compacting, 0 eligible, 16 blocking 2023-10-28 00:00:31,888 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-10-28 00:00:31,888 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-10-28 00:00:31,889 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. because compaction request was cancelled 2023-10-28 00:00:31,889 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-10-28 00:00:31,892 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-10-28 00:00:31,892 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1912): 4904326d2e05286728bb6d35b919e5bf/family2 is initiating minor compaction (all files) 2023-10-28 00:00:31,893 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 4904326d2e05286728bb6d35b919e5bf/family2 in testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:00:31,893 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-974f6211ca1740f2a34fe077aa91bdf2, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-2b1ba95eb4554512b2aa7f4f3bf684a7, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-01571f5665324edb985a32da4c4341a4, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-af19a53e844a4102a55235df865fc030] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/.tmp, totalSize=20.4 K 2023-10-28 00:00:31,894 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-974f6211ca1740f2a34fe077aa91bdf2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698451205463 2023-10-28 00:00:31,894 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-2b1ba95eb4554512b2aa7f4f3bf684a7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698451207431 2023-10-28 00:00:31,895 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-01571f5665324edb985a32da4c4341a4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698451209412 2023-10-28 00:00:31,896 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-af19a53e844a4102a55235df865fc030, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698451211340 2023-10-28 00:00:31,917 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] throttle.PressureAwareThroughputController(145): 4904326d2e05286728bb6d35b919e5bf#family2#compaction#119 average throughput is 0.29 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 57.69 MB/second 2023-10-28 00:00:31,936 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-2bfcfdf09d734e5dac83a9d255bd7680, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-2083b1a0e27a4323afa5cc383672554b, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-4a1c8f8ec1b744a293ebf977fdd7f04d, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-1fc20602194041dda4e593b5943c1c21] to archive 2023-10-28 00:00:31,937 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:31,938 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-2bfcfdf09d734e5dac83a9d255bd7680 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-2bfcfdf09d734e5dac83a9d255bd7680 2023-10-28 00:00:31,939 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-2083b1a0e27a4323afa5cc383672554b to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-2083b1a0e27a4323afa5cc383672554b 2023-10-28 00:00:31,940 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-4a1c8f8ec1b744a293ebf977fdd7f04d to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-4a1c8f8ec1b744a293ebf977fdd7f04d 2023-10-28 00:00:31,941 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-1fc20602194041dda4e593b5943c1c21 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family2/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-1fc20602194041dda4e593b5943c1c21 2023-10-28 00:00:32,007 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:32,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. 2023-10-28 00:00:32,011 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d4d6277130ca600bf099d77a935d3b41, NAME => 'ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41.', STARTKEY => '', ENDKEY => '1'} 2023-10-28 00:00:32,011 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction d4d6277130ca600bf099d77a935d3b41 2023-10-28 00:00:32,011 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-28 00:00:32,011 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for d4d6277130ca600bf099d77a935d3b41 2023-10-28 00:00:32,011 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for d4d6277130ca600bf099d77a935d3b41 2023-10-28 00:00:32,013 INFO [StoreOpener-d4d6277130ca600bf099d77a935d3b41-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d4d6277130ca600bf099d77a935d3b41 2023-10-28 00:00:32,015 DEBUG [StoreOpener-d4d6277130ca600bf099d77a935d3b41-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1 2023-10-28 00:00:32,016 DEBUG [StoreOpener-d4d6277130ca600bf099d77a935d3b41-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1 2023-10-28 00:00:32,016 INFO [StoreOpener-d4d6277130ca600bf099d77a935d3b41-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d4d6277130ca600bf099d77a935d3b41 columnFamilyName family1 2023-10-28 00:00:32,018 INFO [StoreOpener-d4d6277130ca600bf099d77a935d3b41-1] regionserver.HStore(310): Store=d4d6277130ca600bf099d77a935d3b41/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-28 00:00:32,018 INFO [StoreOpener-d4d6277130ca600bf099d77a935d3b41-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region d4d6277130ca600bf099d77a935d3b41 2023-10-28 00:00:32,020 DEBUG [StoreOpener-d4d6277130ca600bf099d77a935d3b41-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2 2023-10-28 00:00:32,020 DEBUG [StoreOpener-d4d6277130ca600bf099d77a935d3b41-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2 2023-10-28 00:00:32,020 INFO [StoreOpener-d4d6277130ca600bf099d77a935d3b41-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d4d6277130ca600bf099d77a935d3b41 columnFamilyName family2 2023-10-28 00:00:32,022 INFO [StoreOpener-d4d6277130ca600bf099d77a935d3b41-1] regionserver.HStore(310): Store=d4d6277130ca600bf099d77a935d3b41/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-28 00:00:32,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41 2023-10-28 00:00:32,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41 2023-10-28 00:00:32,025 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-10-28 00:00:32,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for d4d6277130ca600bf099d77a935d3b41 2023-10-28 00:00:32,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-10-28 00:00:32,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened d4d6277130ca600bf099d77a935d3b41; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70335348, jitterRate=0.04807835817337036}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-10-28 00:00:32,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for d4d6277130ca600bf099d77a935d3b41: 2023-10-28 00:00:32,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41., pid=110, masterSystemTime=1698451232007 2023-10-28 00:00:32,029 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. 2023-10-28 00:00:32,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. 2023-10-28 00:00:32,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. 2023-10-28 00:00:32,029 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f0ae0de72c21217ffb45facabb5a6cd5, NAME => 'ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5.', STARTKEY => '1', ENDKEY => '2'} 2023-10-28 00:00:32,030 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=d4d6277130ca600bf099d77a935d3b41, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:32,030 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction f0ae0de72c21217ffb45facabb5a6cd5 2023-10-28 00:00:32,030 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-28 00:00:32,030 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1698451232030"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451232030"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451232030"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451232030"}]},"ts":"1698451232030"} 2023-10-28 00:00:32,030 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for f0ae0de72c21217ffb45facabb5a6cd5 2023-10-28 00:00:32,030 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for f0ae0de72c21217ffb45facabb5a6cd5 2023-10-28 00:00:32,031 INFO [StoreOpener-f0ae0de72c21217ffb45facabb5a6cd5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f0ae0de72c21217ffb45facabb5a6cd5 2023-10-28 00:00:32,032 DEBUG [StoreOpener-f0ae0de72c21217ffb45facabb5a6cd5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1 2023-10-28 00:00:32,032 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=110, resume processing ppid=106 2023-10-28 00:00:32,032 DEBUG [StoreOpener-f0ae0de72c21217ffb45facabb5a6cd5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1 2023-10-28 00:00:32,033 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=110, ppid=106, state=SUCCESS; OpenRegionProcedure d4d6277130ca600bf099d77a935d3b41, server=jenkins-hbase16.apache.org,43469,1698451096272 in 175 msec 2023-10-28 00:00:32,033 INFO [StoreOpener-f0ae0de72c21217ffb45facabb5a6cd5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f0ae0de72c21217ffb45facabb5a6cd5 columnFamilyName family1 2023-10-28 00:00:32,033 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d4d6277130ca600bf099d77a935d3b41, ASSIGN in 342 msec 2023-10-28 00:00:32,034 INFO [StoreOpener-f0ae0de72c21217ffb45facabb5a6cd5-1] regionserver.HStore(310): Store=f0ae0de72c21217ffb45facabb5a6cd5/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-28 00:00:32,034 INFO [StoreOpener-f0ae0de72c21217ffb45facabb5a6cd5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region f0ae0de72c21217ffb45facabb5a6cd5 2023-10-28 00:00:32,035 DEBUG [StoreOpener-f0ae0de72c21217ffb45facabb5a6cd5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2 2023-10-28 00:00:32,035 DEBUG [StoreOpener-f0ae0de72c21217ffb45facabb5a6cd5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2 2023-10-28 00:00:32,036 INFO [StoreOpener-f0ae0de72c21217ffb45facabb5a6cd5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f0ae0de72c21217ffb45facabb5a6cd5 columnFamilyName family2 2023-10-28 00:00:32,037 INFO [StoreOpener-f0ae0de72c21217ffb45facabb5a6cd5-1] regionserver.HStore(310): Store=f0ae0de72c21217ffb45facabb5a6cd5/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-28 00:00:32,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5 2023-10-28 00:00:32,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5 2023-10-28 00:00:32,039 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-10-28 00:00:32,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for f0ae0de72c21217ffb45facabb5a6cd5 2023-10-28 00:00:32,042 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-10-28 00:00:32,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened f0ae0de72c21217ffb45facabb5a6cd5; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74611861, jitterRate=0.11180336773395538}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-10-28 00:00:32,042 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for f0ae0de72c21217ffb45facabb5a6cd5: 2023-10-28 00:00:32,043 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5., pid=109, masterSystemTime=1698451232007 2023-10-28 00:00:32,044 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. 2023-10-28 00:00:32,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. 2023-10-28 00:00:32,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. 2023-10-28 00:00:32,044 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e6aa79e78506420304e660bae0e1ad9a, NAME => 'ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a.', STARTKEY => '2', ENDKEY => ''} 2023-10-28 00:00:32,044 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=f0ae0de72c21217ffb45facabb5a6cd5, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:32,044 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction e6aa79e78506420304e660bae0e1ad9a 2023-10-28 00:00:32,044 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1698451232044"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451232044"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451232044"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451232044"}]},"ts":"1698451232044"} 2023-10-28 00:00:32,044 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-28 00:00:32,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for e6aa79e78506420304e660bae0e1ad9a 2023-10-28 00:00:32,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for e6aa79e78506420304e660bae0e1ad9a 2023-10-28 00:00:32,046 INFO [StoreOpener-e6aa79e78506420304e660bae0e1ad9a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region e6aa79e78506420304e660bae0e1ad9a 2023-10-28 00:00:32,047 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=109, resume processing ppid=107 2023-10-28 00:00:32,047 DEBUG [StoreOpener-e6aa79e78506420304e660bae0e1ad9a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1 2023-10-28 00:00:32,047 DEBUG [StoreOpener-e6aa79e78506420304e660bae0e1ad9a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1 2023-10-28 00:00:32,047 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=109, ppid=107, state=SUCCESS; OpenRegionProcedure f0ae0de72c21217ffb45facabb5a6cd5, server=jenkins-hbase16.apache.org,43469,1698451096272 in 192 msec 2023-10-28 00:00:32,047 INFO [StoreOpener-e6aa79e78506420304e660bae0e1ad9a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e6aa79e78506420304e660bae0e1ad9a columnFamilyName family1 2023-10-28 00:00:32,048 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=f0ae0de72c21217ffb45facabb5a6cd5, ASSIGN in 356 msec 2023-10-28 00:00:32,048 INFO [StoreOpener-e6aa79e78506420304e660bae0e1ad9a-1] regionserver.HStore(310): Store=e6aa79e78506420304e660bae0e1ad9a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-28 00:00:32,048 INFO [StoreOpener-e6aa79e78506420304e660bae0e1ad9a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region e6aa79e78506420304e660bae0e1ad9a 2023-10-28 00:00:32,049 DEBUG [StoreOpener-e6aa79e78506420304e660bae0e1ad9a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2 2023-10-28 00:00:32,050 DEBUG [StoreOpener-e6aa79e78506420304e660bae0e1ad9a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2 2023-10-28 00:00:32,050 INFO [StoreOpener-e6aa79e78506420304e660bae0e1ad9a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e6aa79e78506420304e660bae0e1ad9a columnFamilyName family2 2023-10-28 00:00:32,051 INFO [StoreOpener-e6aa79e78506420304e660bae0e1ad9a-1] regionserver.HStore(310): Store=e6aa79e78506420304e660bae0e1ad9a/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-28 00:00:32,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a 2023-10-28 00:00:32,052 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a 2023-10-28 00:00:32,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-10-28 00:00:32,054 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for e6aa79e78506420304e660bae0e1ad9a 2023-10-28 00:00:32,056 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-10-28 00:00:32,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened e6aa79e78506420304e660bae0e1ad9a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60912669, jitterRate=-0.09233050048351288}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-10-28 00:00:32,056 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for e6aa79e78506420304e660bae0e1ad9a: 2023-10-28 00:00:32,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a., pid=111, masterSystemTime=1698451232007 2023-10-28 00:00:32,058 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. 2023-10-28 00:00:32,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. 2023-10-28 00:00:32,059 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=e6aa79e78506420304e660bae0e1ad9a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:32,059 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1698451232059"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451232059"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451232059"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451232059"}]},"ts":"1698451232059"} 2023-10-28 00:00:32,062 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=111, resume processing ppid=108 2023-10-28 00:00:32,062 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=111, ppid=108, state=SUCCESS; OpenRegionProcedure e6aa79e78506420304e660bae0e1ad9a, server=jenkins-hbase16.apache.org,43469,1698451096272 in 204 msec 2023-10-28 00:00:32,063 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=108, resume processing ppid=105 2023-10-28 00:00:32,063 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=e6aa79e78506420304e660bae0e1ad9a, ASSIGN in 371 msec 2023-10-28 00:00:32,064 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-10-28 00:00:32,064 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451232064"}]},"ts":"1698451232064"} 2023-10-28 00:00:32,065 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-10-28 00:00:32,100 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-10-28 00:00:32,103 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 894 msec 2023-10-28 00:00:32,318 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-10-28 00:00:32,319 INFO [Listener at localhost/40295] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-10-28 00:00:32,328 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/.tmp/family1/5bb42eb7a5a449519937922934c546dc as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/5bb42eb7a5a449519937922934c546dc 2023-10-28 00:00:32,329 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:00:32,330 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:00:32,331 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:00:32,335 INFO [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in 795595fcd155993336d014aba73423ea/family1 of 795595fcd155993336d014aba73423ea into 5bb42eb7a5a449519937922934c546dc(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-10-28 00:00:32,335 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 795595fcd155993336d014aba73423ea: 2023-10-28 00:00:32,335 INFO [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1698451231043; duration=1sec 2023-10-28 00:00:32,335 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-28 00:00:32,335 DEBUG [RS:0;jenkins-hbase16:43469-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-28 00:00:32,336 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-10-28 00:00:32,336 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-10-28 00:00:32,337 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:32,337 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-10-28 00:00:32,337 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-10-28 00:00:32,337 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:32,337 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-10-28 00:00:32,349 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:32,349 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:32,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:32,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:32,349 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:32,349 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-28 00:00:32,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-10-28 00:00:32,350 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:32,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-28 00:00:32,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-10-28 00:00:32,350 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-10-28 00:00:32,351 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-10-28 00:00:32,351 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-10-28 00:00:32,351 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:32,351 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-10-28 00:00:32,351 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. 2023-10-28 00:00:32,351 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-10-28 00:00:32,351 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-28 00:00:32,351 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. 2023-10-28 00:00:32,351 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. 2023-10-28 00:00:32,351 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. started... 2023-10-28 00:00:32,351 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. started... 2023-10-28 00:00:32,351 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. started... 2023-10-28 00:00:32,353 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing e6aa79e78506420304e660bae0e1ad9a 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-10-28 00:00:32,353 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f0ae0de72c21217ffb45facabb5a6cd5 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-10-28 00:00:32,353 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing d4d6277130ca600bf099d77a935d3b41 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-10-28 00:00:32,430 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-dcf9afa91e314f178c59ced5a25fa944, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-30493979777049ce8ebd4cf3bc0cd5d5, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-2cb1c76d629141179060550b5919816b, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-1ac14e9e9ba54d8f8eae2adbb5a965e9, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/e4c492e9769a4c46a68b0ec6a5c0e6ad_SeqId_34_] to archive 2023-10-28 00:00:32,431 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:32,435 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-dcf9afa91e314f178c59ced5a25fa944 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-dcf9afa91e314f178c59ced5a25fa944 2023-10-28 00:00:32,438 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-30493979777049ce8ebd4cf3bc0cd5d5 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-30493979777049ce8ebd4cf3bc0cd5d5 2023-10-28 00:00:32,441 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-2cb1c76d629141179060550b5919816b to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-2cb1c76d629141179060550b5919816b 2023-10-28 00:00:32,442 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-1ac14e9e9ba54d8f8eae2adbb5a965e9 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/testCompactionWithHFileLink=795595fcd155993336d014aba73423ea-1ac14e9e9ba54d8f8eae2adbb5a965e9 2023-10-28 00:00:32,444 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/e4c492e9769a4c46a68b0ec6a5c0e6ad_SeqId_34_ to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/family1/e4c492e9769a4c46a68b0ec6a5c0e6ad_SeqId_34_ 2023-10-28 00:00:32,748 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/.tmp/family2/b706187ec10843fcaad383ce1fbfb0ec as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/b706187ec10843fcaad383ce1fbfb0ec 2023-10-28 00:00:32,755 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 4904326d2e05286728bb6d35b919e5bf/family2 of 4904326d2e05286728bb6d35b919e5bf into b706187ec10843fcaad383ce1fbfb0ec(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-10-28 00:00:32,755 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 4904326d2e05286728bb6d35b919e5bf: 2023-10-28 00:00:32,755 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf., storeName=4904326d2e05286728bb6d35b919e5bf/family2, priority=12, startTime=1698451229419; duration=0sec 2023-10-28 00:00:32,756 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-28 00:00:32,767 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202310280698adc416c74353b88291ba29af19fb to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202310280698adc416c74353b88291ba29af19fb 2023-10-28 00:00:32,767 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202310285338510605a34930ab701bdea8408e02 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202310285338510605a34930ab701bdea8408e02 2023-10-28 00:00:32,767 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231028b846b25b82bd4230b142100a15339a6d to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20231028b846b25b82bd4230b142100a15339a6d 2023-10-28 00:00:32,771 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/.tmp/family1/a47a938c09824cf08c3079b0d15fac36 2023-10-28 00:00:32,829 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-974f6211ca1740f2a34fe077aa91bdf2, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-2b1ba95eb4554512b2aa7f4f3bf684a7, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-01571f5665324edb985a32da4c4341a4, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-af19a53e844a4102a55235df865fc030] to archive 2023-10-28 00:00:32,830 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:32,834 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-974f6211ca1740f2a34fe077aa91bdf2 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-974f6211ca1740f2a34fe077aa91bdf2 2023-10-28 00:00:32,837 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-2b1ba95eb4554512b2aa7f4f3bf684a7 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-2b1ba95eb4554512b2aa7f4f3bf684a7 2023-10-28 00:00:32,840 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-01571f5665324edb985a32da4c4341a4 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-01571f5665324edb985a32da4c4341a4 2023-10-28 00:00:32,843 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-af19a53e844a4102a55235df865fc030 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/family2/testCompactionWithHFileLink=4904326d2e05286728bb6d35b919e5bf-af19a53e844a4102a55235df865fc030 2023-10-28 00:00:33,172 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/.tmp/family1/08348291d3514227984c5d4cb47af3dd 2023-10-28 00:00:33,172 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/.tmp/family1/ddb0686a042d4584ac8851895511fe0a 2023-10-28 00:00:33,185 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231028125836a14434427c9ca9c09a5a1a80eb to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20231028125836a14434427c9ca9c09a5a1a80eb 2023-10-28 00:00:33,195 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202310287f487938b945457b86347ba8984aea17 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202310287f487938b945457b86347ba8984aea17 2023-10-28 00:00:33,592 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/.tmp/family2/b44dc3e7313449b697f31d53a210ca12 2023-10-28 00:00:33,600 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/.tmp/family2/99daa2f47e6a485c819427228247aaf6 2023-10-28 00:00:33,601 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023102803335423343440408f89281c37c9eeb6 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023102803335423343440408f89281c37c9eeb6 2023-10-28 00:00:33,602 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/.tmp/family1/a47a938c09824cf08c3079b0d15fac36 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/a47a938c09824cf08c3079b0d15fac36 2023-10-28 00:00:33,607 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/.tmp/family1/ddb0686a042d4584ac8851895511fe0a as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/ddb0686a042d4584ac8851895511fe0a 2023-10-28 00:00:33,607 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/.tmp/family2/859793a3fb8a443b987cec63223bc081 2023-10-28 00:00:33,607 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/a47a938c09824cf08c3079b0d15fac36, entries=4, sequenceid=5, filesize=5.4 K 2023-10-28 00:00:33,608 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/.tmp/family2/b44dc3e7313449b697f31d53a210ca12 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/b44dc3e7313449b697f31d53a210ca12 2023-10-28 00:00:33,615 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/ddb0686a042d4584ac8851895511fe0a, entries=4, sequenceid=5, filesize=5.4 K 2023-10-28 00:00:33,616 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/.tmp/family1/08348291d3514227984c5d4cb47af3dd as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/08348291d3514227984c5d4cb47af3dd 2023-10-28 00:00:33,616 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/.tmp/family2/99daa2f47e6a485c819427228247aaf6 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/99daa2f47e6a485c819427228247aaf6 2023-10-28 00:00:33,617 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/b44dc3e7313449b697f31d53a210ca12, entries=2, sequenceid=5, filesize=5.1 K 2023-10-28 00:00:33,618 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for e6aa79e78506420304e660bae0e1ad9a in 1265ms, sequenceid=5, compaction requested=false 2023-10-28 00:00:33,618 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-10-28 00:00:33,619 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for e6aa79e78506420304e660bae0e1ad9a: 2023-10-28 00:00:33,619 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. 2023-10-28 00:00:33,619 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-10-28 00:00:33,620 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/99daa2f47e6a485c819427228247aaf6, entries=2, sequenceid=5, filesize=5.1 K 2023-10-28 00:00:33,620 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/08348291d3514227984c5d4cb47af3dd, entries=4, sequenceid=5, filesize=5.4 K 2023-10-28 00:00:33,620 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for d4d6277130ca600bf099d77a935d3b41 in 1267ms, sequenceid=5, compaction requested=false 2023-10-28 00:00:33,620 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/.tmp/family2/859793a3fb8a443b987cec63223bc081 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/859793a3fb8a443b987cec63223bc081 2023-10-28 00:00:33,621 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for d4d6277130ca600bf099d77a935d3b41: 2023-10-28 00:00:33,621 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. 2023-10-28 00:00:33,621 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-10-28 00:00:33,625 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/859793a3fb8a443b987cec63223bc081, entries=2, sequenceid=5, filesize=5.1 K 2023-10-28 00:00:33,626 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for f0ae0de72c21217ffb45facabb5a6cd5 in 1273ms, sequenceid=5, compaction requested=false 2023-10-28 00:00:33,626 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f0ae0de72c21217ffb45facabb5a6cd5: 2023-10-28 00:00:33,626 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. 2023-10-28 00:00:33,626 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-10-28 00:00:33,626 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-10-28 00:00:33,626 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:33,626 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-10-28 00:00:33,626 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-10-28 00:00:33,657 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:33,657 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:33,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:33,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:33,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:33,657 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:33,658 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-28 00:00:33,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:33,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:33,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:33,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:33,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:33,660 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-10-28 00:00:33,660 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@31ae0120[Count = 0] remaining members to acquire global barrier 2023-10-28 00:00:33,660 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-10-28 00:00:33,660 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:33,665 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:33,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:33,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:33,666 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-10-28 00:00:33,666 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-10-28 00:00:33,666 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-28 00:00:33,666 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:33,666 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-28 00:00:33,674 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-10-28 00:00:33,674 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:33,674 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-10-28 00:00:33,674 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:33,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:33,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:33,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:33,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:33,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:33,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:33,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:33,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:33,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:33,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:33,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-28 00:00:33,679 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-10-28 00:00:33,679 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-10-28 00:00:33,679 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-28 00:00:33,680 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-28 00:00:33,680 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-10-28 00:00:33,680 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-28 00:00:33,690 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:33,691 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:33,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:33,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:33,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:33,691 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:33,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:33,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:33,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:33,692 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:33,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:33,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:33,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:33,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:33,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:33,695 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:33,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:33,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:33,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:33,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:33,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:33,707 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:33,707 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-10-28 00:00:33,707 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-10-28 00:00:33,707 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:33,707 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:33,707 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:33,707 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:33,708 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:33,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:33,707 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:33,707 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-10-28 00:00:33,708 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-28 00:00:33,708 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-10-28 00:00:33,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:33,708 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:33,708 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-10-28 00:00:33,709 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-28 00:00:33,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:33,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:33,809 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-28 00:00:33,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-28 00:00:33,815 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:00:33,816 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:00:33,816 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:00:33,821 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-28 00:00:33,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-10-28 00:00:33,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-10-28 00:00:33,824 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-10-28 00:00:33,824 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:33,824 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-10-28 00:00:33,824 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-10-28 00:00:33,825 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:33,825 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-10-28 00:00:33,832 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:33,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:33,832 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:33,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:33,833 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:33,834 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-28 00:00:33,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-10-28 00:00:33,835 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:33,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-28 00:00:33,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-10-28 00:00:33,836 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-10-28 00:00:33,836 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-10-28 00:00:33,836 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-10-28 00:00:33,836 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-10-28 00:00:33,837 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:33,837 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-10-28 00:00:33,837 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. 2023-10-28 00:00:33,837 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. 2023-10-28 00:00:33,837 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-10-28 00:00:33,838 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-28 00:00:33,837 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. started... 2023-10-28 00:00:33,837 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. started... 2023-10-28 00:00:33,837 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. 2023-10-28 00:00:33,838 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f0ae0de72c21217ffb45facabb5a6cd5 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-10-28 00:00:33,838 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing d4d6277130ca600bf099d77a935d3b41 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-10-28 00:00:33,838 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. started... 2023-10-28 00:00:33,839 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing e6aa79e78506420304e660bae0e1ad9a 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-10-28 00:00:34,269 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202310289d7ba9baf0894699b634977e39fd51d8 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202310289d7ba9baf0894699b634977e39fd51d8 2023-10-28 00:00:34,269 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202310280982b073b18c4c4d806f1ee2131d8cd0 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202310280982b073b18c4c4d806f1ee2131d8cd0 2023-10-28 00:00:34,269 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023102811c43ab9443b4ca78b5c37b756684d7a to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023102811c43ab9443b4ca78b5c37b756684d7a 2023-10-28 00:00:34,677 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/.tmp/family1/cf063eb7efcb432b9269f975d72b7ea0 2023-10-28 00:00:34,678 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/.tmp/family1/41f9b81e2da645a7bd4cf033d48abf3c 2023-10-28 00:00:34,681 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/.tmp/family1/cdf2a09df2ac480495b54d748d38cbe3 2023-10-28 00:00:35,105 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023102847abb09ff1f1452e8a26398833b3605b to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023102847abb09ff1f1452e8a26398833b3605b 2023-10-28 00:00:35,106 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231028243e5d9fd7454604b0e566d561f6ac4c to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20231028243e5d9fd7454604b0e566d561f6ac4c 2023-10-28 00:00:35,106 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231028627eddd085e84be5aeae6e4fe0efb5a0 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20231028627eddd085e84be5aeae6e4fe0efb5a0 2023-10-28 00:00:35,514 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/.tmp/family2/1544245cd4aa41bf9c2b467b74a80dd8 2023-10-28 00:00:35,519 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/.tmp/family2/84d887495ba143639bfefadb94df11c5 2023-10-28 00:00:35,519 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/.tmp/family2/8eef1a0f367c4731960b5e6a76b8f1ec 2023-10-28 00:00:35,524 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/.tmp/family1/cf063eb7efcb432b9269f975d72b7ea0 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/cf063eb7efcb432b9269f975d72b7ea0 2023-10-28 00:00:35,527 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/.tmp/family1/cdf2a09df2ac480495b54d748d38cbe3 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/cdf2a09df2ac480495b54d748d38cbe3 2023-10-28 00:00:35,527 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/.tmp/family1/41f9b81e2da645a7bd4cf033d48abf3c as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/41f9b81e2da645a7bd4cf033d48abf3c 2023-10-28 00:00:35,531 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/cf063eb7efcb432b9269f975d72b7ea0, entries=4, sequenceid=9, filesize=5.4 K 2023-10-28 00:00:35,531 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/41f9b81e2da645a7bd4cf033d48abf3c, entries=4, sequenceid=9, filesize=5.4 K 2023-10-28 00:00:35,532 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/.tmp/family2/1544245cd4aa41bf9c2b467b74a80dd8 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/1544245cd4aa41bf9c2b467b74a80dd8 2023-10-28 00:00:35,533 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/.tmp/family2/8eef1a0f367c4731960b5e6a76b8f1ec as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/8eef1a0f367c4731960b5e6a76b8f1ec 2023-10-28 00:00:35,533 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/cdf2a09df2ac480495b54d748d38cbe3, entries=4, sequenceid=9, filesize=5.4 K 2023-10-28 00:00:35,534 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/.tmp/family2/84d887495ba143639bfefadb94df11c5 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/84d887495ba143639bfefadb94df11c5 2023-10-28 00:00:35,536 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/8eef1a0f367c4731960b5e6a76b8f1ec, entries=2, sequenceid=9, filesize=5.1 K 2023-10-28 00:00:35,536 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/1544245cd4aa41bf9c2b467b74a80dd8, entries=2, sequenceid=9, filesize=5.1 K 2023-10-28 00:00:35,536 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/84d887495ba143639bfefadb94df11c5, entries=2, sequenceid=9, filesize=5.1 K 2023-10-28 00:00:35,537 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for e6aa79e78506420304e660bae0e1ad9a in 1698ms, sequenceid=9, compaction requested=false 2023-10-28 00:00:35,537 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for d4d6277130ca600bf099d77a935d3b41 in 1699ms, sequenceid=9, compaction requested=false 2023-10-28 00:00:35,537 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for e6aa79e78506420304e660bae0e1ad9a: 2023-10-28 00:00:35,537 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for d4d6277130ca600bf099d77a935d3b41: 2023-10-28 00:00:35,537 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for f0ae0de72c21217ffb45facabb5a6cd5 in 1699ms, sequenceid=9, compaction requested=false 2023-10-28 00:00:35,537 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. 2023-10-28 00:00:35,537 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. 2023-10-28 00:00:35,537 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-10-28 00:00:35,537 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f0ae0de72c21217ffb45facabb5a6cd5: 2023-10-28 00:00:35,537 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-10-28 00:00:35,537 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. 2023-10-28 00:00:35,537 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-10-28 00:00:35,537 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-10-28 00:00:35,537 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:35,537 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-10-28 00:00:35,537 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-10-28 00:00:35,591 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:35,591 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:35,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:35,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:35,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:35,591 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:35,592 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-28 00:00:35,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:35,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:35,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:35,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:35,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:35,596 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-10-28 00:00:35,596 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3c0901c6[Count = 0] remaining members to acquire global barrier 2023-10-28 00:00:35,596 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-10-28 00:00:35,596 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:35,607 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:35,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:35,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:35,608 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-10-28 00:00:35,608 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-10-28 00:00:35,608 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-28 00:00:35,608 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:35,608 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-28 00:00:35,616 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-10-28 00:00:35,616 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:35,616 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:35,616 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-10-28 00:00:35,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:35,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:35,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:35,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:35,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:35,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:35,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:35,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:35,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:35,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:35,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-28 00:00:35,625 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-10-28 00:00:35,625 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-10-28 00:00:35,625 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-28 00:00:35,625 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-28 00:00:35,625 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-10-28 00:00:35,625 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-28 00:00:35,632 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:35,632 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:35,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:35,632 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:35,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:35,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:35,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:35,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:35,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:35,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:35,634 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:35,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:35,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:35,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:35,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:35,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:35,637 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:35,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:35,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:35,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:35,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:35,649 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:35,649 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-10-28 00:00:35,649 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-10-28 00:00:35,649 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:35,649 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:35,649 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-10-28 00:00:35,649 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-28 00:00:35,649 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-10-28 00:00:35,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:35,649 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:35,650 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-10-28 00:00:35,650 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-28 00:00:35,649 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:35,649 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:35,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:35,650 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:35,650 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:35,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:35,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:35,750 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-28 00:00:35,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-28 00:00:35,755 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:00:35,755 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:00:35,756 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:00:35,763 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-28 00:00:35,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-10-28 00:00:35,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-10-28 00:00:35,766 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-10-28 00:00:35,766 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:35,766 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-10-28 00:00:35,766 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-10-28 00:00:35,767 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:35,767 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-10-28 00:00:35,774 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:35,774 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:35,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:35,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:35,774 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:35,774 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-28 00:00:35,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-10-28 00:00:35,774 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:35,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-28 00:00:35,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-10-28 00:00:35,775 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-10-28 00:00:35,775 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-10-28 00:00:35,775 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-10-28 00:00:35,775 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-10-28 00:00:35,775 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:35,775 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-10-28 00:00:35,775 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. 2023-10-28 00:00:35,775 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. started... 2023-10-28 00:00:35,775 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. 2023-10-28 00:00:35,775 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. started... 2023-10-28 00:00:35,775 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f0ae0de72c21217ffb45facabb5a6cd5 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-10-28 00:00:35,775 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing d4d6277130ca600bf099d77a935d3b41 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-10-28 00:00:35,776 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-10-28 00:00:35,776 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-28 00:00:35,776 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. 2023-10-28 00:00:35,776 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. started... 2023-10-28 00:00:35,776 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing e6aa79e78506420304e660bae0e1ad9a 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-10-28 00:00:35,794 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202310284c072374307447a0abf3dab423932988 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202310284c072374307447a0abf3dab423932988 2023-10-28 00:00:36,196 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202310288ea79a915c164b88ba7ce74feb854176 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202310288ea79a915c164b88ba7ce74feb854176 2023-10-28 00:00:36,202 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/.tmp/family1/d9fd97690ac349a4b57d9fd379ae84f7 2023-10-28 00:00:36,203 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231028889638b6e7e746a7b74d58cbeadebc10 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20231028889638b6e7e746a7b74d58cbeadebc10 2023-10-28 00:00:36,204 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/.tmp/family1/0fd15e34f3a146caa6e806df35586bff 2023-10-28 00:00:36,208 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/.tmp/family1/3dc9caf1249a48a8a708524b594b3bb3 2023-10-28 00:00:36,226 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202310283b6f390d77854c6abc44d66d144dc08c to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202310283b6f390d77854c6abc44d66d144dc08c 2023-10-28 00:00:36,631 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/.tmp/family2/00c4192dc71241f3a3ccc6786c2b22f9 2023-10-28 00:00:36,633 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231028dbf70a9680a14b6aab790551f10fb263 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20231028dbf70a9680a14b6aab790551f10fb263 2023-10-28 00:00:36,633 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023102810e3d9de63154be4be0846663e0f6fcb to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c2023102810e3d9de63154be4be0846663e0f6fcb 2023-10-28 00:00:36,636 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/.tmp/family1/3dc9caf1249a48a8a708524b594b3bb3 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/3dc9caf1249a48a8a708524b594b3bb3 2023-10-28 00:00:36,637 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/.tmp/family2/e41251a1ed574389b710a3071296f9e5 2023-10-28 00:00:36,640 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/3dc9caf1249a48a8a708524b594b3bb3, entries=4, sequenceid=13, filesize=5.4 K 2023-10-28 00:00:36,640 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/.tmp/family1/d9fd97690ac349a4b57d9fd379ae84f7 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/d9fd97690ac349a4b57d9fd379ae84f7 2023-10-28 00:00:36,640 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/.tmp/family2/00c4192dc71241f3a3ccc6786c2b22f9 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/00c4192dc71241f3a3ccc6786c2b22f9 2023-10-28 00:00:36,643 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/d9fd97690ac349a4b57d9fd379ae84f7, entries=4, sequenceid=13, filesize=5.4 K 2023-10-28 00:00:36,644 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/00c4192dc71241f3a3ccc6786c2b22f9, entries=2, sequenceid=13, filesize=5.1 K 2023-10-28 00:00:36,644 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/.tmp/family2/e41251a1ed574389b710a3071296f9e5 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/e41251a1ed574389b710a3071296f9e5 2023-10-28 00:00:36,644 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for d4d6277130ca600bf099d77a935d3b41 in 869ms, sequenceid=13, compaction requested=true 2023-10-28 00:00:36,644 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for d4d6277130ca600bf099d77a935d3b41: 2023-10-28 00:00:36,644 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. 2023-10-28 00:00:36,644 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-10-28 00:00:36,647 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/e41251a1ed574389b710a3071296f9e5, entries=2, sequenceid=13, filesize=5.1 K 2023-10-28 00:00:36,647 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for e6aa79e78506420304e660bae0e1ad9a in 871ms, sequenceid=13, compaction requested=true 2023-10-28 00:00:36,647 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for e6aa79e78506420304e660bae0e1ad9a: 2023-10-28 00:00:36,648 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. 2023-10-28 00:00:36,648 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-10-28 00:00:37,038 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/.tmp/family2/7781bef57f7e4c78a45dd36481a3cef0 2023-10-28 00:00:37,047 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/.tmp/family1/0fd15e34f3a146caa6e806df35586bff as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/0fd15e34f3a146caa6e806df35586bff 2023-10-28 00:00:37,055 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/0fd15e34f3a146caa6e806df35586bff, entries=4, sequenceid=13, filesize=5.4 K 2023-10-28 00:00:37,057 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/.tmp/family2/7781bef57f7e4c78a45dd36481a3cef0 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/7781bef57f7e4c78a45dd36481a3cef0 2023-10-28 00:00:37,065 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/7781bef57f7e4c78a45dd36481a3cef0, entries=2, sequenceid=13, filesize=5.1 K 2023-10-28 00:00:37,066 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for f0ae0de72c21217ffb45facabb5a6cd5 in 1291ms, sequenceid=13, compaction requested=true 2023-10-28 00:00:37,066 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f0ae0de72c21217ffb45facabb5a6cd5: 2023-10-28 00:00:37,066 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. 2023-10-28 00:00:37,066 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-10-28 00:00:37,066 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-10-28 00:00:37,066 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:37,066 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-10-28 00:00:37,066 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-10-28 00:00:37,099 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:37,099 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:37,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:37,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:37,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:37,100 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:37,100 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-28 00:00:37,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:37,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:37,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:37,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:37,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:37,104 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-10-28 00:00:37,104 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5b8985b9[Count = 0] remaining members to acquire global barrier 2023-10-28 00:00:37,104 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-10-28 00:00:37,104 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:37,115 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:37,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:37,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:37,116 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-10-28 00:00:37,116 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-10-28 00:00:37,116 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-28 00:00:37,116 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:37,117 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-28 00:00:37,124 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-10-28 00:00:37,124 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:37,124 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:37,124 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-10-28 00:00:37,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:37,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:37,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:37,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:37,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:37,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:37,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:37,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:37,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:37,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:37,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-28 00:00:37,134 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-10-28 00:00:37,134 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-10-28 00:00:37,134 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-28 00:00:37,134 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-28 00:00:37,134 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-10-28 00:00:37,134 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-28 00:00:37,140 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:37,140 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:37,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:37,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:37,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:37,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:37,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:37,141 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:37,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:37,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:37,142 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:37,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:37,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:37,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:37,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:37,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:37,146 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:37,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:37,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:37,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:37,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:37,157 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:37,157 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:37,157 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:37,157 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-10-28 00:00:37,157 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:37,157 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-10-28 00:00:37,157 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-28 00:00:37,157 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-10-28 00:00:37,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:37,157 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:37,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:37,157 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:37,157 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-10-28 00:00:37,157 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:37,157 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-28 00:00:37,158 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:37,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:37,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:37,258 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-28 00:00:37,259 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-28 00:00:37,260 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:00:37,260 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:00:37,260 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:00:37,264 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-28 00:00:37,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-10-28 00:00:37,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-10-28 00:00:37,266 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-10-28 00:00:37,266 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:37,266 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-10-28 00:00:37,266 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-10-28 00:00:37,267 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:37,267 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-10-28 00:00:37,274 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:37,274 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:37,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:37,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:37,274 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:37,274 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-28 00:00:37,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-10-28 00:00:37,275 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:37,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-28 00:00:37,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-10-28 00:00:37,275 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-10-28 00:00:37,276 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-10-28 00:00:37,276 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-10-28 00:00:37,276 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-10-28 00:00:37,276 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:37,276 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-10-28 00:00:37,276 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. 2023-10-28 00:00:37,276 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. started... 2023-10-28 00:00:37,276 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. 2023-10-28 00:00:37,276 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. started... 2023-10-28 00:00:37,276 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f0ae0de72c21217ffb45facabb5a6cd5 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-10-28 00:00:37,276 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing d4d6277130ca600bf099d77a935d3b41 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-10-28 00:00:37,276 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-10-28 00:00:37,276 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-28 00:00:37,276 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. 2023-10-28 00:00:37,276 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. started... 2023-10-28 00:00:37,276 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing e6aa79e78506420304e660bae0e1ad9a 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-10-28 00:00:37,516 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-10-28 00:00:37,693 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202310284cbf6171ede949b9907a2d4e6da22b9a to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202310284cbf6171ede949b9907a2d4e6da22b9a 2023-10-28 00:00:37,693 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202310287392c054ee7846caa703c1f762ad04d1 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202310287392c054ee7846caa703c1f762ad04d1 2023-10-28 00:00:37,693 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231028d19797f04f704b00b431df1887d1937f to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20231028d19797f04f704b00b431df1887d1937f 2023-10-28 00:00:38,099 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/.tmp/family1/2da786c2c1994e689a23b4cc3165d211 2023-10-28 00:00:38,100 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/.tmp/family1/cf90e29aaac94051a55f933993264f92 2023-10-28 00:00:38,100 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/.tmp/family1/ec62fa7a1bac4823878274c960496c0f 2023-10-28 00:00:38,522 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202310284ebc10100d1c4009ad304f79c4d80f2f to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202310284ebc10100d1c4009ad304f79c4d80f2f 2023-10-28 00:00:38,522 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023102839c75b33380143b79b0c4cba8a6ed46b to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023102839c75b33380143b79b0c4cba8a6ed46b 2023-10-28 00:00:38,522 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231028579c76d661d5461e97cb93fe9344c0ae to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20231028579c76d661d5461e97cb93fe9344c0ae 2023-10-28 00:00:38,929 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/.tmp/family2/b7d74a02b99f41e8a5286b1aaaa59662 2023-10-28 00:00:38,929 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/.tmp/family2/454d1caf5c4c4085a84f125e238741e1 2023-10-28 00:00:38,930 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/.tmp/family2/e1c74fa3721f483d8433f0c1edd7f270 2023-10-28 00:00:38,940 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/.tmp/family1/2da786c2c1994e689a23b4cc3165d211 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/2da786c2c1994e689a23b4cc3165d211 2023-10-28 00:00:38,940 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/.tmp/family1/cf90e29aaac94051a55f933993264f92 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/cf90e29aaac94051a55f933993264f92 2023-10-28 00:00:38,940 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/.tmp/family1/ec62fa7a1bac4823878274c960496c0f as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/ec62fa7a1bac4823878274c960496c0f 2023-10-28 00:00:38,944 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/cf90e29aaac94051a55f933993264f92, entries=4, sequenceid=17, filesize=5.4 K 2023-10-28 00:00:38,944 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/ec62fa7a1bac4823878274c960496c0f, entries=4, sequenceid=17, filesize=5.4 K 2023-10-28 00:00:38,944 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/2da786c2c1994e689a23b4cc3165d211, entries=4, sequenceid=17, filesize=5.4 K 2023-10-28 00:00:38,945 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/.tmp/family2/454d1caf5c4c4085a84f125e238741e1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/454d1caf5c4c4085a84f125e238741e1 2023-10-28 00:00:38,945 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/.tmp/family2/e1c74fa3721f483d8433f0c1edd7f270 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/e1c74fa3721f483d8433f0c1edd7f270 2023-10-28 00:00:38,945 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/.tmp/family2/b7d74a02b99f41e8a5286b1aaaa59662 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/b7d74a02b99f41e8a5286b1aaaa59662 2023-10-28 00:00:38,950 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/b7d74a02b99f41e8a5286b1aaaa59662, entries=2, sequenceid=17, filesize=5.1 K 2023-10-28 00:00:38,950 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/e1c74fa3721f483d8433f0c1edd7f270, entries=2, sequenceid=17, filesize=5.1 K 2023-10-28 00:00:38,950 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/454d1caf5c4c4085a84f125e238741e1, entries=2, sequenceid=17, filesize=5.1 K 2023-10-28 00:00:38,951 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for d4d6277130ca600bf099d77a935d3b41 in 1675ms, sequenceid=17, compaction requested=true 2023-10-28 00:00:38,951 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for e6aa79e78506420304e660bae0e1ad9a in 1675ms, sequenceid=17, compaction requested=true 2023-10-28 00:00:38,951 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for f0ae0de72c21217ffb45facabb5a6cd5 in 1675ms, sequenceid=17, compaction requested=true 2023-10-28 00:00:38,951 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for e6aa79e78506420304e660bae0e1ad9a: 2023-10-28 00:00:38,951 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. 2023-10-28 00:00:38,951 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for d4d6277130ca600bf099d77a935d3b41: 2023-10-28 00:00:38,951 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-10-28 00:00:38,951 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f0ae0de72c21217ffb45facabb5a6cd5: 2023-10-28 00:00:38,951 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. 2023-10-28 00:00:38,951 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. 2023-10-28 00:00:38,951 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-10-28 00:00:38,951 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-10-28 00:00:38,951 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-10-28 00:00:38,951 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:38,951 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-10-28 00:00:38,951 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-10-28 00:00:38,982 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:38,982 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:38,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:38,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:38,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:38,983 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:38,983 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-28 00:00:38,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:38,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:38,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:38,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:38,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:38,987 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-10-28 00:00:38,988 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7133e995[Count = 0] remaining members to acquire global barrier 2023-10-28 00:00:38,988 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-10-28 00:00:38,988 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:38,999 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:38,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:38,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:38,999 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-10-28 00:00:39,000 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-10-28 00:00:39,000 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-28 00:00:39,000 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:39,000 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-28 00:00:39,007 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-10-28 00:00:39,007 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:39,007 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-10-28 00:00:39,007 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:39,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:39,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:39,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:39,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:39,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:39,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:39,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:39,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:39,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:39,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:39,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-28 00:00:39,018 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-10-28 00:00:39,018 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-10-28 00:00:39,018 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-28 00:00:39,018 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-28 00:00:39,018 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-10-28 00:00:39,018 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-28 00:00:39,030 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:39,030 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:39,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:39,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:39,030 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:39,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:39,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:39,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:39,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:39,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:39,031 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:39,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:39,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:39,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:39,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:39,035 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:39,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:39,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:39,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:39,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:39,048 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:39,048 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:39,048 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:39,048 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-10-28 00:00:39,049 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-28 00:00:39,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:39,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:39,048 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:39,049 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-10-28 00:00:39,048 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-10-28 00:00:39,049 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-10-28 00:00:39,049 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-10-28 00:00:39,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:39,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:39,049 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:39,049 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-28 00:00:39,049 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:39,049 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:39,049 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:39,149 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-28 00:00:39,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-28 00:00:39,163 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-10-28 00:00:39,169 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-10-28 00:00:39,175 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-10-28 00:00:39,176 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-10-28 00:00:39,206 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-28 00:00:39,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-10-28 00:00:39,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-10-28 00:00:39,208 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-10-28 00:00:39,208 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:39,208 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-10-28 00:00:39,208 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-10-28 00:00:39,209 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:39,209 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-10-28 00:00:39,215 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:39,215 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:39,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:39,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:39,216 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:39,216 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-28 00:00:39,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-10-28 00:00:39,217 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:39,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-28 00:00:39,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-10-28 00:00:39,217 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-10-28 00:00:39,217 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-10-28 00:00:39,218 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-10-28 00:00:39,218 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-10-28 00:00:39,218 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:39,218 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-10-28 00:00:39,218 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. 2023-10-28 00:00:39,218 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. started... 2023-10-28 00:00:39,218 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. 2023-10-28 00:00:39,218 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. started... 2023-10-28 00:00:39,218 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. 2023-10-28 00:00:39,218 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-10-28 00:00:39,219 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-28 00:00:39,218 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. started... 2023-10-28 00:00:39,218 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing d4d6277130ca600bf099d77a935d3b41 2/2 column families, dataSize=138 B heapSize=944 B 2023-10-28 00:00:39,218 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f0ae0de72c21217ffb45facabb5a6cd5 2/2 column families, dataSize=138 B heapSize=944 B 2023-10-28 00:00:39,219 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing e6aa79e78506420304e660bae0e1ad9a 2/2 column families, dataSize=138 B heapSize=944 B 2023-10-28 00:00:40,045 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/.tmp/family1/f022d4067aeb414293b2ae918364f18b 2023-10-28 00:00:40,046 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/.tmp/family1/05b0a3e5076341b7a01b2ce274629cf5 2023-10-28 00:00:40,046 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/.tmp/family1/31dfc9d58fe04fb0b26cf7d9a157591d 2023-10-28 00:00:40,053 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f022d4067aeb414293b2ae918364f18b 2023-10-28 00:00:40,054 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 05b0a3e5076341b7a01b2ce274629cf5 2023-10-28 00:00:40,054 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 31dfc9d58fe04fb0b26cf7d9a157591d 2023-10-28 00:00:40,875 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/.tmp/family2/2a8fade72e3f444383cdc53914ba9a56 2023-10-28 00:00:40,876 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/.tmp/family2/4d785ea0e49b471cb35ba6741928fd04 2023-10-28 00:00:40,876 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/.tmp/family2/c6558c09d46c4cd4b15b39747d7aab55 2023-10-28 00:00:40,886 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c6558c09d46c4cd4b15b39747d7aab55 2023-10-28 00:00:40,886 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4d785ea0e49b471cb35ba6741928fd04 2023-10-28 00:00:40,886 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2a8fade72e3f444383cdc53914ba9a56 2023-10-28 00:00:40,886 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/.tmp/family1/31dfc9d58fe04fb0b26cf7d9a157591d as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/31dfc9d58fe04fb0b26cf7d9a157591d 2023-10-28 00:00:40,886 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/.tmp/family1/05b0a3e5076341b7a01b2ce274629cf5 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/05b0a3e5076341b7a01b2ce274629cf5 2023-10-28 00:00:40,887 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/.tmp/family1/f022d4067aeb414293b2ae918364f18b as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/f022d4067aeb414293b2ae918364f18b 2023-10-28 00:00:40,890 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 31dfc9d58fe04fb0b26cf7d9a157591d 2023-10-28 00:00:40,890 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 05b0a3e5076341b7a01b2ce274629cf5 2023-10-28 00:00:40,890 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f022d4067aeb414293b2ae918364f18b 2023-10-28 00:00:40,890 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/05b0a3e5076341b7a01b2ce274629cf5, entries=3, sequenceid=23, filesize=5.0 K 2023-10-28 00:00:40,890 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/31dfc9d58fe04fb0b26cf7d9a157591d, entries=3, sequenceid=23, filesize=5.0 K 2023-10-28 00:00:40,890 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/f022d4067aeb414293b2ae918364f18b, entries=3, sequenceid=23, filesize=5.0 K 2023-10-28 00:00:40,891 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/.tmp/family2/c6558c09d46c4cd4b15b39747d7aab55 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/c6558c09d46c4cd4b15b39747d7aab55 2023-10-28 00:00:40,891 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/.tmp/family2/4d785ea0e49b471cb35ba6741928fd04 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/4d785ea0e49b471cb35ba6741928fd04 2023-10-28 00:00:40,891 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/.tmp/family2/2a8fade72e3f444383cdc53914ba9a56 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/2a8fade72e3f444383cdc53914ba9a56 2023-10-28 00:00:40,895 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c6558c09d46c4cd4b15b39747d7aab55 2023-10-28 00:00:40,895 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2a8fade72e3f444383cdc53914ba9a56 2023-10-28 00:00:40,895 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/c6558c09d46c4cd4b15b39747d7aab55, entries=1, sequenceid=23, filesize=5.0 K 2023-10-28 00:00:40,895 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/2a8fade72e3f444383cdc53914ba9a56, entries=1, sequenceid=23, filesize=5.0 K 2023-10-28 00:00:40,896 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for d4d6277130ca600bf099d77a935d3b41 in 1678ms, sequenceid=23, compaction requested=true 2023-10-28 00:00:40,896 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for e6aa79e78506420304e660bae0e1ad9a in 1677ms, sequenceid=23, compaction requested=true 2023-10-28 00:00:40,896 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for d4d6277130ca600bf099d77a935d3b41: 2023-10-28 00:00:40,896 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for e6aa79e78506420304e660bae0e1ad9a: 2023-10-28 00:00:40,896 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. 2023-10-28 00:00:40,896 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. 2023-10-28 00:00:40,896 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-10-28 00:00:40,896 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-10-28 00:00:40,898 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4d785ea0e49b471cb35ba6741928fd04 2023-10-28 00:00:40,898 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/4d785ea0e49b471cb35ba6741928fd04, entries=1, sequenceid=23, filesize=5.0 K 2023-10-28 00:00:40,898 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for f0ae0de72c21217ffb45facabb5a6cd5 in 1680ms, sequenceid=23, compaction requested=true 2023-10-28 00:00:40,899 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f0ae0de72c21217ffb45facabb5a6cd5: 2023-10-28 00:00:40,899 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. 2023-10-28 00:00:40,899 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-10-28 00:00:40,899 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-10-28 00:00:40,899 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:40,899 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-10-28 00:00:40,899 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-10-28 00:00:40,907 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:40,907 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:40,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:40,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:40,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:40,907 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:40,907 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-28 00:00:40,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:40,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:40,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:40,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:40,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:40,908 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-10-28 00:00:40,908 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5315800f[Count = 0] remaining members to acquire global barrier 2023-10-28 00:00:40,908 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-10-28 00:00:40,908 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:40,934 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:40,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:40,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:40,934 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:40,934 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-28 00:00:40,934 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-10-28 00:00:40,934 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-10-28 00:00:40,934 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-28 00:00:40,947 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-10-28 00:00:40,947 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:40,947 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:40,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:40,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:40,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:40,947 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-10-28 00:00:40,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:40,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:40,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:40,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:40,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:40,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:40,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:40,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-28 00:00:40,949 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-10-28 00:00:40,949 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-10-28 00:00:40,949 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-28 00:00:40,949 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-28 00:00:40,949 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-10-28 00:00:40,949 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-28 00:00:40,957 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:40,957 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:40,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:40,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:40,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:40,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:40,957 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:40,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:40,957 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:40,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:40,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:40,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:40,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:40,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:40,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:40,958 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:40,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:40,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:40,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:40,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-10-28 00:00:40,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:40,965 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:40,965 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:40,965 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:40,965 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-10-28 00:00:40,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:40,965 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-10-28 00:00:40,965 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-28 00:00:40,965 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:40,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:40,965 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-10-28 00:00:40,965 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:40,965 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:40,965 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-10-28 00:00:40,965 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-10-28 00:00:40,965 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-10-28 00:00:40,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:40,965 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-10-28 00:00:40,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:40,966 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-28 00:00:41,066 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-28 00:00:41,067 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-28 00:00:41,067 DEBUG [Listener at localhost/40295] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-10-28 00:00:41,067 DEBUG [Listener at localhost/40295] regionserver.HStore(1912): f0ae0de72c21217ffb45facabb5a6cd5/family1 is initiating major compaction (all files) 2023-10-28 00:00:41,068 INFO [Listener at localhost/40295] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-10-28 00:00:41,068 INFO [Listener at localhost/40295] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-10-28 00:00:41,068 INFO [Listener at localhost/40295] regionserver.HRegion(2259): Starting compaction of f0ae0de72c21217ffb45facabb5a6cd5/family1 in ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. 2023-10-28 00:00:41,068 INFO [Listener at localhost/40295] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/08348291d3514227984c5d4cb47af3dd, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/cdf2a09df2ac480495b54d748d38cbe3, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/0fd15e34f3a146caa6e806df35586bff, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/ec62fa7a1bac4823878274c960496c0f, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/05b0a3e5076341b7a01b2ce274629cf5] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/.tmp, totalSize=26.6 K 2023-10-28 00:00:41,069 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 08348291d3514227984c5d4cb47af3dd, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698451232330 2023-10-28 00:00:41,069 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting cdf2a09df2ac480495b54d748d38cbe3, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698451233816 2023-10-28 00:00:41,070 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 0fd15e34f3a146caa6e806df35586bff, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698451235755 2023-10-28 00:00:41,070 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting ec62fa7a1bac4823878274c960496c0f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698451237260 2023-10-28 00:00:41,071 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 05b0a3e5076341b7a01b2ce274629cf5, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-10-28 00:00:41,087 INFO [Listener at localhost/40295] throttle.PressureAwareThroughputController(145): f0ae0de72c21217ffb45facabb5a6cd5#family1#compaction#150 average throughput is 0.33 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-10-28 00:00:41,505 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20231028b7cb062c9e4e495f859d290aaf428b7c_del 2023-10-28 00:00:41,505 INFO [Listener at localhost/40295] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231028b7cb062c9e4e495f859d290aaf428b7c_del to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20231028b7cb062c9e4e495f859d290aaf428b7c_del 2023-10-28 00:00:42,335 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 45030120bd9948ae86ecf7810b50c480 2023-10-28 00:00:42,337 DEBUG [Listener at localhost/40295] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/.tmp/family1/45030120bd9948ae86ecf7810b50c480 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/45030120bd9948ae86ecf7810b50c480 2023-10-28 00:00:42,344 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 45030120bd9948ae86ecf7810b50c480 2023-10-28 00:00:42,346 INFO [Listener at localhost/40295] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in f0ae0de72c21217ffb45facabb5a6cd5/family1 of f0ae0de72c21217ffb45facabb5a6cd5 into 45030120bd9948ae86ecf7810b50c480(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-10-28 00:00:42,346 DEBUG [Listener at localhost/40295] regionserver.HRegion(2289): Compaction status journal for f0ae0de72c21217ffb45facabb5a6cd5: 2023-10-28 00:00:42,346 DEBUG [Listener at localhost/40295] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-10-28 00:00:42,346 DEBUG [Listener at localhost/40295] regionserver.HStore(1912): f0ae0de72c21217ffb45facabb5a6cd5/family2 is initiating major compaction (all files) 2023-10-28 00:00:42,346 INFO [Listener at localhost/40295] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-10-28 00:00:42,346 INFO [Listener at localhost/40295] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-10-28 00:00:42,346 INFO [Listener at localhost/40295] regionserver.HRegion(2259): Starting compaction of f0ae0de72c21217ffb45facabb5a6cd5/family2 in ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. 2023-10-28 00:00:42,347 INFO [Listener at localhost/40295] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/859793a3fb8a443b987cec63223bc081, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/84d887495ba143639bfefadb94df11c5, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/7781bef57f7e4c78a45dd36481a3cef0, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/e1c74fa3721f483d8433f0c1edd7f270, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/4d785ea0e49b471cb35ba6741928fd04] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/.tmp, totalSize=25.3 K 2023-10-28 00:00:42,347 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 859793a3fb8a443b987cec63223bc081, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698451232330 2023-10-28 00:00:42,348 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 84d887495ba143639bfefadb94df11c5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698451233816 2023-10-28 00:00:42,348 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 7781bef57f7e4c78a45dd36481a3cef0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698451235755 2023-10-28 00:00:42,349 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting e1c74fa3721f483d8433f0c1edd7f270, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698451237260 2023-10-28 00:00:42,349 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 4d785ea0e49b471cb35ba6741928fd04, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-10-28 00:00:42,362 INFO [Listener at localhost/40295] throttle.PressureAwareThroughputController(145): f0ae0de72c21217ffb45facabb5a6cd5#family2#compaction#151 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-10-28 00:00:42,430 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/08348291d3514227984c5d4cb47af3dd, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/cdf2a09df2ac480495b54d748d38cbe3, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/0fd15e34f3a146caa6e806df35586bff, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/ec62fa7a1bac4823878274c960496c0f, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/05b0a3e5076341b7a01b2ce274629cf5] to archive 2023-10-28 00:00:42,433 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:42,439 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/08348291d3514227984c5d4cb47af3dd to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/08348291d3514227984c5d4cb47af3dd 2023-10-28 00:00:42,443 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/cdf2a09df2ac480495b54d748d38cbe3 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/cdf2a09df2ac480495b54d748d38cbe3 2023-10-28 00:00:42,447 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/0fd15e34f3a146caa6e806df35586bff to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/0fd15e34f3a146caa6e806df35586bff 2023-10-28 00:00:42,450 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/ec62fa7a1bac4823878274c960496c0f to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/ec62fa7a1bac4823878274c960496c0f 2023-10-28 00:00:42,453 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/05b0a3e5076341b7a01b2ce274629cf5 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/05b0a3e5076341b7a01b2ce274629cf5 2023-10-28 00:00:42,777 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20231028da53ab5a5ed7480da05fdae8f6ba15c7_del 2023-10-28 00:00:42,778 INFO [Listener at localhost/40295] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231028da53ab5a5ed7480da05fdae8f6ba15c7_del to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20231028da53ab5a5ed7480da05fdae8f6ba15c7_del 2023-10-28 00:00:43,608 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f5f2491ca62b40fa8bce71ba4c42f797 2023-10-28 00:00:43,610 DEBUG [Listener at localhost/40295] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/.tmp/family2/f5f2491ca62b40fa8bce71ba4c42f797 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/f5f2491ca62b40fa8bce71ba4c42f797 2023-10-28 00:00:43,614 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f5f2491ca62b40fa8bce71ba4c42f797 2023-10-28 00:00:43,615 INFO [Listener at localhost/40295] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in f0ae0de72c21217ffb45facabb5a6cd5/family2 of f0ae0de72c21217ffb45facabb5a6cd5 into f5f2491ca62b40fa8bce71ba4c42f797(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-10-28 00:00:43,615 DEBUG [Listener at localhost/40295] regionserver.HRegion(2289): Compaction status journal for f0ae0de72c21217ffb45facabb5a6cd5: 2023-10-28 00:00:43,616 DEBUG [Listener at localhost/40295] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-10-28 00:00:43,616 DEBUG [Listener at localhost/40295] regionserver.HStore(1912): d4d6277130ca600bf099d77a935d3b41/family1 is initiating major compaction (all files) 2023-10-28 00:00:43,616 INFO [Listener at localhost/40295] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-10-28 00:00:43,616 INFO [Listener at localhost/40295] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-10-28 00:00:43,616 INFO [Listener at localhost/40295] regionserver.HRegion(2259): Starting compaction of d4d6277130ca600bf099d77a935d3b41/family1 in ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. 2023-10-28 00:00:43,616 INFO [Listener at localhost/40295] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/ddb0686a042d4584ac8851895511fe0a, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/cf063eb7efcb432b9269f975d72b7ea0, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/3dc9caf1249a48a8a708524b594b3bb3, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/2da786c2c1994e689a23b4cc3165d211, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/f022d4067aeb414293b2ae918364f18b] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/.tmp, totalSize=26.6 K 2023-10-28 00:00:43,617 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting ddb0686a042d4584ac8851895511fe0a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698451232329 2023-10-28 00:00:43,617 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting cf063eb7efcb432b9269f975d72b7ea0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698451233815 2023-10-28 00:00:43,617 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 3dc9caf1249a48a8a708524b594b3bb3, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698451235755 2023-10-28 00:00:43,618 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 2da786c2c1994e689a23b4cc3165d211, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698451237260 2023-10-28 00:00:43,618 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting f022d4067aeb414293b2ae918364f18b, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-10-28 00:00:43,627 INFO [Listener at localhost/40295] throttle.PressureAwareThroughputController(145): d4d6277130ca600bf099d77a935d3b41#family1#compaction#152 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-10-28 00:00:43,630 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/859793a3fb8a443b987cec63223bc081, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/84d887495ba143639bfefadb94df11c5, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/7781bef57f7e4c78a45dd36481a3cef0, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/e1c74fa3721f483d8433f0c1edd7f270, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/4d785ea0e49b471cb35ba6741928fd04] to archive 2023-10-28 00:00:43,631 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:43,632 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/859793a3fb8a443b987cec63223bc081 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/859793a3fb8a443b987cec63223bc081 2023-10-28 00:00:43,633 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/84d887495ba143639bfefadb94df11c5 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/84d887495ba143639bfefadb94df11c5 2023-10-28 00:00:43,634 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/7781bef57f7e4c78a45dd36481a3cef0 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/7781bef57f7e4c78a45dd36481a3cef0 2023-10-28 00:00:43,635 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/e1c74fa3721f483d8433f0c1edd7f270 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/e1c74fa3721f483d8433f0c1edd7f270 2023-10-28 00:00:43,636 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/4d785ea0e49b471cb35ba6741928fd04 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family2/4d785ea0e49b471cb35ba6741928fd04 2023-10-28 00:00:44,044 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20231028d4b41f538d694f179e94d4f6d5811eef_del 2023-10-28 00:00:44,044 INFO [Listener at localhost/40295] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231028d4b41f538d694f179e94d4f6d5811eef_del to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20231028d4b41f538d694f179e94d4f6d5811eef_del 2023-10-28 00:00:44,871 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c8a7043ef95c493891cbf8b1ad83ff2c 2023-10-28 00:00:44,872 DEBUG [Listener at localhost/40295] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/.tmp/family1/c8a7043ef95c493891cbf8b1ad83ff2c as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/c8a7043ef95c493891cbf8b1ad83ff2c 2023-10-28 00:00:44,877 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c8a7043ef95c493891cbf8b1ad83ff2c 2023-10-28 00:00:44,878 INFO [Listener at localhost/40295] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d4d6277130ca600bf099d77a935d3b41/family1 of d4d6277130ca600bf099d77a935d3b41 into c8a7043ef95c493891cbf8b1ad83ff2c(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-10-28 00:00:44,878 DEBUG [Listener at localhost/40295] regionserver.HRegion(2289): Compaction status journal for d4d6277130ca600bf099d77a935d3b41: 2023-10-28 00:00:44,878 DEBUG [Listener at localhost/40295] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-10-28 00:00:44,879 DEBUG [Listener at localhost/40295] regionserver.HStore(1912): d4d6277130ca600bf099d77a935d3b41/family2 is initiating major compaction (all files) 2023-10-28 00:00:44,879 INFO [Listener at localhost/40295] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-10-28 00:00:44,879 INFO [Listener at localhost/40295] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-10-28 00:00:44,879 INFO [Listener at localhost/40295] regionserver.HRegion(2259): Starting compaction of d4d6277130ca600bf099d77a935d3b41/family2 in ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. 2023-10-28 00:00:44,879 INFO [Listener at localhost/40295] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/99daa2f47e6a485c819427228247aaf6, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/1544245cd4aa41bf9c2b467b74a80dd8, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/00c4192dc71241f3a3ccc6786c2b22f9, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/b7d74a02b99f41e8a5286b1aaaa59662, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/2a8fade72e3f444383cdc53914ba9a56] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/.tmp, totalSize=25.3 K 2023-10-28 00:00:44,880 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 99daa2f47e6a485c819427228247aaf6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698451232329 2023-10-28 00:00:44,880 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 1544245cd4aa41bf9c2b467b74a80dd8, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698451233815 2023-10-28 00:00:44,881 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 00c4192dc71241f3a3ccc6786c2b22f9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698451235755 2023-10-28 00:00:44,882 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting b7d74a02b99f41e8a5286b1aaaa59662, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698451237260 2023-10-28 00:00:44,882 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 2a8fade72e3f444383cdc53914ba9a56, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-10-28 00:00:44,895 INFO [Listener at localhost/40295] throttle.PressureAwareThroughputController(145): d4d6277130ca600bf099d77a935d3b41#family2#compaction#153 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-10-28 00:00:44,905 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20231028a030c8ed1eda4bd8912a45793478cf7c_del 2023-10-28 00:00:44,905 INFO [Listener at localhost/40295] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231028a030c8ed1eda4bd8912a45793478cf7c_del to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20231028a030c8ed1eda4bd8912a45793478cf7c_del 2023-10-28 00:00:44,932 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/ddb0686a042d4584ac8851895511fe0a, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/cf063eb7efcb432b9269f975d72b7ea0, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/3dc9caf1249a48a8a708524b594b3bb3, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/2da786c2c1994e689a23b4cc3165d211, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/f022d4067aeb414293b2ae918364f18b] to archive 2023-10-28 00:00:44,933 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:44,937 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/ddb0686a042d4584ac8851895511fe0a to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/ddb0686a042d4584ac8851895511fe0a 2023-10-28 00:00:44,940 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/cf063eb7efcb432b9269f975d72b7ea0 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/cf063eb7efcb432b9269f975d72b7ea0 2023-10-28 00:00:44,943 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/3dc9caf1249a48a8a708524b594b3bb3 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/3dc9caf1249a48a8a708524b594b3bb3 2023-10-28 00:00:44,946 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/2da786c2c1994e689a23b4cc3165d211 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/2da786c2c1994e689a23b4cc3165d211 2023-10-28 00:00:44,948 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/f022d4067aeb414293b2ae918364f18b to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/f022d4067aeb414293b2ae918364f18b 2023-10-28 00:00:45,331 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2fc325ecf8a94421add8a3f5bd2b3ae5 2023-10-28 00:00:45,333 DEBUG [Listener at localhost/40295] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/.tmp/family2/2fc325ecf8a94421add8a3f5bd2b3ae5 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/2fc325ecf8a94421add8a3f5bd2b3ae5 2023-10-28 00:00:45,341 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2fc325ecf8a94421add8a3f5bd2b3ae5 2023-10-28 00:00:45,342 INFO [Listener at localhost/40295] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d4d6277130ca600bf099d77a935d3b41/family2 of d4d6277130ca600bf099d77a935d3b41 into 2fc325ecf8a94421add8a3f5bd2b3ae5(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-10-28 00:00:45,342 DEBUG [Listener at localhost/40295] regionserver.HRegion(2289): Compaction status journal for d4d6277130ca600bf099d77a935d3b41: 2023-10-28 00:00:45,342 DEBUG [Listener at localhost/40295] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-10-28 00:00:45,343 DEBUG [Listener at localhost/40295] regionserver.HStore(1912): e6aa79e78506420304e660bae0e1ad9a/family1 is initiating major compaction (all files) 2023-10-28 00:00:45,343 INFO [Listener at localhost/40295] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-10-28 00:00:45,343 INFO [Listener at localhost/40295] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-10-28 00:00:45,343 INFO [Listener at localhost/40295] regionserver.HRegion(2259): Starting compaction of e6aa79e78506420304e660bae0e1ad9a/family1 in ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. 2023-10-28 00:00:45,343 INFO [Listener at localhost/40295] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/a47a938c09824cf08c3079b0d15fac36, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/41f9b81e2da645a7bd4cf033d48abf3c, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/d9fd97690ac349a4b57d9fd379ae84f7, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/cf90e29aaac94051a55f933993264f92, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/31dfc9d58fe04fb0b26cf7d9a157591d] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/.tmp, totalSize=26.6 K 2023-10-28 00:00:45,344 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting a47a938c09824cf08c3079b0d15fac36, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698451232331 2023-10-28 00:00:45,345 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 41f9b81e2da645a7bd4cf033d48abf3c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698451233816 2023-10-28 00:00:45,345 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting d9fd97690ac349a4b57d9fd379ae84f7, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698451235756 2023-10-28 00:00:45,346 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting cf90e29aaac94051a55f933993264f92, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698451237260 2023-10-28 00:00:45,346 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 31dfc9d58fe04fb0b26cf7d9a157591d, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-10-28 00:00:45,356 INFO [Listener at localhost/40295] throttle.PressureAwareThroughputController(145): e6aa79e78506420304e660bae0e1ad9a#family1#compaction#154 average throughput is 0.55 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-10-28 00:00:45,364 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023102861020868b0504f69b98d32673561e467_del 2023-10-28 00:00:45,364 INFO [Listener at localhost/40295] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023102861020868b0504f69b98d32673561e467_del to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023102861020868b0504f69b98d32673561e467_del 2023-10-28 00:00:45,429 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/99daa2f47e6a485c819427228247aaf6, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/1544245cd4aa41bf9c2b467b74a80dd8, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/00c4192dc71241f3a3ccc6786c2b22f9, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/b7d74a02b99f41e8a5286b1aaaa59662, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/2a8fade72e3f444383cdc53914ba9a56] to archive 2023-10-28 00:00:45,431 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:45,437 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/99daa2f47e6a485c819427228247aaf6 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/99daa2f47e6a485c819427228247aaf6 2023-10-28 00:00:45,440 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/1544245cd4aa41bf9c2b467b74a80dd8 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/1544245cd4aa41bf9c2b467b74a80dd8 2023-10-28 00:00:45,442 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/00c4192dc71241f3a3ccc6786c2b22f9 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/00c4192dc71241f3a3ccc6786c2b22f9 2023-10-28 00:00:45,444 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/b7d74a02b99f41e8a5286b1aaaa59662 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/b7d74a02b99f41e8a5286b1aaaa59662 2023-10-28 00:00:45,446 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/2a8fade72e3f444383cdc53914ba9a56 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family2/2a8fade72e3f444383cdc53914ba9a56 2023-10-28 00:00:46,202 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1764790f58e24b189f01ffd876e301dc 2023-10-28 00:00:46,204 DEBUG [Listener at localhost/40295] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/.tmp/family1/1764790f58e24b189f01ffd876e301dc as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/1764790f58e24b189f01ffd876e301dc 2023-10-28 00:00:46,213 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1764790f58e24b189f01ffd876e301dc 2023-10-28 00:00:46,214 INFO [Listener at localhost/40295] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in e6aa79e78506420304e660bae0e1ad9a/family1 of e6aa79e78506420304e660bae0e1ad9a into 1764790f58e24b189f01ffd876e301dc(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-10-28 00:00:46,214 DEBUG [Listener at localhost/40295] regionserver.HRegion(2289): Compaction status journal for e6aa79e78506420304e660bae0e1ad9a: 2023-10-28 00:00:46,214 DEBUG [Listener at localhost/40295] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-10-28 00:00:46,214 DEBUG [Listener at localhost/40295] regionserver.HStore(1912): e6aa79e78506420304e660bae0e1ad9a/family2 is initiating major compaction (all files) 2023-10-28 00:00:46,214 INFO [Listener at localhost/40295] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-10-28 00:00:46,214 INFO [Listener at localhost/40295] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-10-28 00:00:46,214 INFO [Listener at localhost/40295] regionserver.HRegion(2259): Starting compaction of e6aa79e78506420304e660bae0e1ad9a/family2 in ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. 2023-10-28 00:00:46,215 INFO [Listener at localhost/40295] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/b44dc3e7313449b697f31d53a210ca12, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/8eef1a0f367c4731960b5e6a76b8f1ec, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/e41251a1ed574389b710a3071296f9e5, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/454d1caf5c4c4085a84f125e238741e1, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/c6558c09d46c4cd4b15b39747d7aab55] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/.tmp, totalSize=25.3 K 2023-10-28 00:00:46,215 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting b44dc3e7313449b697f31d53a210ca12, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698451232331 2023-10-28 00:00:46,216 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 8eef1a0f367c4731960b5e6a76b8f1ec, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698451233816 2023-10-28 00:00:46,216 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting e41251a1ed574389b710a3071296f9e5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698451235756 2023-10-28 00:00:46,217 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 454d1caf5c4c4085a84f125e238741e1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698451237260 2023-10-28 00:00:46,217 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting c6558c09d46c4cd4b15b39747d7aab55, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-10-28 00:00:46,229 INFO [Listener at localhost/40295] throttle.PressureAwareThroughputController(145): e6aa79e78506420304e660bae0e1ad9a#family2#compaction#155 average throughput is 0.25 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-10-28 00:00:46,229 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/a47a938c09824cf08c3079b0d15fac36, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/41f9b81e2da645a7bd4cf033d48abf3c, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/d9fd97690ac349a4b57d9fd379ae84f7, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/cf90e29aaac94051a55f933993264f92, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/31dfc9d58fe04fb0b26cf7d9a157591d] to archive 2023-10-28 00:00:46,230 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:46,233 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/a47a938c09824cf08c3079b0d15fac36 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/a47a938c09824cf08c3079b0d15fac36 2023-10-28 00:00:46,234 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/41f9b81e2da645a7bd4cf033d48abf3c to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/41f9b81e2da645a7bd4cf033d48abf3c 2023-10-28 00:00:46,235 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/d9fd97690ac349a4b57d9fd379ae84f7 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/d9fd97690ac349a4b57d9fd379ae84f7 2023-10-28 00:00:46,236 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/cf90e29aaac94051a55f933993264f92 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/cf90e29aaac94051a55f933993264f92 2023-10-28 00:00:46,237 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/31dfc9d58fe04fb0b26cf7d9a157591d to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/31dfc9d58fe04fb0b26cf7d9a157591d 2023-10-28 00:00:46,645 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202310284d20437bd2aa4dbd94b675a0fef1d17d_del 2023-10-28 00:00:46,646 INFO [Listener at localhost/40295] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202310284d20437bd2aa4dbd94b675a0fef1d17d_del to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202310284d20437bd2aa4dbd94b675a0fef1d17d_del 2023-10-28 00:00:47,476 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f6987c5946b3438ea7bb27e99b5808bb 2023-10-28 00:00:47,477 DEBUG [Listener at localhost/40295] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/.tmp/family2/f6987c5946b3438ea7bb27e99b5808bb as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/f6987c5946b3438ea7bb27e99b5808bb 2023-10-28 00:00:47,482 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f6987c5946b3438ea7bb27e99b5808bb 2023-10-28 00:00:47,483 INFO [Listener at localhost/40295] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in e6aa79e78506420304e660bae0e1ad9a/family2 of e6aa79e78506420304e660bae0e1ad9a into f6987c5946b3438ea7bb27e99b5808bb(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-10-28 00:00:47,484 DEBUG [Listener at localhost/40295] regionserver.HRegion(2289): Compaction status journal for e6aa79e78506420304e660bae0e1ad9a: 2023-10-28 00:00:47,489 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-10-28 00:00:47,493 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-10-28 00:00:47,497 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-10-28 00:00:47,498 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-10-28 00:00:47,503 INFO [Listener at localhost/40295] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-10-28 00:00:47,521 INFO [Listener at localhost/40295] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-10-28 00:00:47,525 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20231028d4b41f538d694f179e94d4f6d5811eef_del 2023-10-28 00:00:47,537 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-10-28 00:00:47,537 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20231028b7cb062c9e4e495f859d290aaf428b7c_del 2023-10-28 00:00:47,538 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x22171f55 to 127.0.0.1:57369 2023-10-28 00:00:47,538 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/b44dc3e7313449b697f31d53a210ca12, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/8eef1a0f367c4731960b5e6a76b8f1ec, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/e41251a1ed574389b710a3071296f9e5, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/454d1caf5c4c4085a84f125e238741e1, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/c6558c09d46c4cd4b15b39747d7aab55] to archive 2023-10-28 00:00:47,538 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-10-28 00:00:47,538 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-10-28 00:00:47,539 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6136aaf5 to 127.0.0.1:57369 2023-10-28 00:00:47,539 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-10-28 00:00:47,539 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:47,540 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/b44dc3e7313449b697f31d53a210ca12 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/b44dc3e7313449b697f31d53a210ca12 2023-10-28 00:00:47,541 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/8eef1a0f367c4731960b5e6a76b8f1ec to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/8eef1a0f367c4731960b5e6a76b8f1ec 2023-10-28 00:00:47,541 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023102861020868b0504f69b98d32673561e467_del 2023-10-28 00:00:47,542 INFO [Listener at localhost/40295] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-10-28 00:00:47,542 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/e41251a1ed574389b710a3071296f9e5 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/e41251a1ed574389b710a3071296f9e5 2023-10-28 00:00:47,543 DEBUG [Listener at localhost/40295] zookeeper.ReadOnlyZKClient(139): Connect 0x39c29c94 to 127.0.0.1:57369 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-10-28 00:00:47,543 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/454d1caf5c4c4085a84f125e238741e1 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/454d1caf5c4c4085a84f125e238741e1 2023-10-28 00:00:47,545 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/c6558c09d46c4cd4b15b39747d7aab55 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family2/c6558c09d46c4cd4b15b39747d7aab55 2023-10-28 00:00:47,583 DEBUG [Listener at localhost/40295] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@52ec3e9c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-28 00:00:47,585 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20231028 for table='ns:testMinorCompaction' and column='family1' 2023-10-28 00:00:47,585 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20231028 for table='ns:testMinorCompaction' and column='family1' 2023-10-28 00:00:47,585 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20231028 for table='ns:testMinorCompaction' and column='family1' 2023-10-28 00:00:48,012 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231028086786cac2a44257adae2dcc0414e742 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20231028086786cac2a44257adae2dcc0414e742 2023-10-28 00:00:48,013 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231028195d99ef530448c2b04545c67510f1d4 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20231028195d99ef530448c2b04545c67510f1d4 2023-10-28 00:00:48,013 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231028 table='ns:testMinorCompaction' column='family1' 2023-10-28 00:00:48,013 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231028 table='ns:testMinorCompaction' column='family1' 2023-10-28 00:00:48,025 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-10-28 00:00:48,027 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:35844, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-10-28 00:00:48,032 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-10-28 00:00:48,039 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:54310, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-10-28 00:00:48,056 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20231028/family1/2e4eebca101f4d54b2de8ad83ad9cf55 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-10-28 00:00:48,056 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20231028/family1/6e2ad66f87514fb58203d27da5eec7e1 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-10-28 00:00:48,057 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20231028/family1/2e4eebca101f4d54b2de8ad83ad9cf55}] 2023-10-28 00:00:48,057 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20231028/family1/6e2ad66f87514fb58203d27da5eec7e1}] 2023-10-28 00:00:48,061 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20231028/family1/6e2ad66f87514fb58203d27da5eec7e1 for inclusion in f0ae0de72c21217ffb45facabb5a6cd5/family1 2023-10-28 00:00:48,061 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20231028/family1/2e4eebca101f4d54b2de8ad83ad9cf55 for inclusion in d4d6277130ca600bf099d77a935d3b41/family1 2023-10-28 00:00:48,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-10-28 00:00:48,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-10-28 00:00:48,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(777): Region bounds: first= last=1 2023-10-28 00:00:48,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(777): Region bounds: first=1 last=2 2023-10-28 00:00:48,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for d4d6277130ca600bf099d77a935d3b41: 2023-10-28 00:00:48,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for f0ae0de72c21217ffb45facabb5a6cd5: 2023-10-28 00:00:48,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20231028/family1/2e4eebca101f4d54b2de8ad83ad9cf55 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__ns_testMinorCompaction__20nmpgvadikm80aheegrqaa5bvrfdfnrcv5c6smc8l78pcp3fen3m3v2fmmqfplf/family1/2e4eebca101f4d54b2de8ad83ad9cf55 2023-10-28 00:00:48,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20231028/family1/6e2ad66f87514fb58203d27da5eec7e1 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__ns_testMinorCompaction__s98k0fae0gu5f3kgifkpkpopa2kh5136ark192bqc4d13siuk0r885ll8f9nv3cm/family1/6e2ad66f87514fb58203d27da5eec7e1 2023-10-28 00:00:48,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__ns_testMinorCompaction__s98k0fae0gu5f3kgifkpkpopa2kh5136ark192bqc4d13siuk0r885ll8f9nv3cm/family1/6e2ad66f87514fb58203d27da5eec7e1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/175df235943945128ff777e9b342ce98_SeqId_28_ 2023-10-28 00:00:48,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__ns_testMinorCompaction__20nmpgvadikm80aheegrqaa5bvrfdfnrcv5c6smc8l78pcp3fen3m3v2fmmqfplf/family1/2e4eebca101f4d54b2de8ad83ad9cf55 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/f14c82d720aa4fe1984702ebdf78c818_SeqId_28_ 2023-10-28 00:00:48,066 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__ns_testMinorCompaction__s98k0fae0gu5f3kgifkpkpopa2kh5136ark192bqc4d13siuk0r885ll8f9nv3cm/family1/6e2ad66f87514fb58203d27da5eec7e1 into f0ae0de72c21217ffb45facabb5a6cd5/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/175df235943945128ff777e9b342ce98_SeqId_28_ - updating store file list. 2023-10-28 00:00:48,067 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__ns_testMinorCompaction__20nmpgvadikm80aheegrqaa5bvrfdfnrcv5c6smc8l78pcp3fen3m3v2fmmqfplf/family1/2e4eebca101f4d54b2de8ad83ad9cf55 into d4d6277130ca600bf099d77a935d3b41/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/f14c82d720aa4fe1984702ebdf78c818_SeqId_28_ - updating store file list. 2023-10-28 00:00:48,070 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/175df235943945128ff777e9b342ce98_SeqId_28_ into f0ae0de72c21217ffb45facabb5a6cd5/family1 2023-10-28 00:00:48,070 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/f14c82d720aa4fe1984702ebdf78c818_SeqId_28_ into d4d6277130ca600bf099d77a935d3b41/family1 2023-10-28 00:00:48,070 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__ns_testMinorCompaction__s98k0fae0gu5f3kgifkpkpopa2kh5136ark192bqc4d13siuk0r885ll8f9nv3cm/family1/6e2ad66f87514fb58203d27da5eec7e1 into f0ae0de72c21217ffb45facabb5a6cd5/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/family1/175df235943945128ff777e9b342ce98_SeqId_28_) 2023-10-28 00:00:48,070 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__ns_testMinorCompaction__20nmpgvadikm80aheegrqaa5bvrfdfnrcv5c6smc8l78pcp3fen3m3v2fmmqfplf/family1/2e4eebca101f4d54b2de8ad83ad9cf55 into d4d6277130ca600bf099d77a935d3b41/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/family1/f14c82d720aa4fe1984702ebdf78c818_SeqId_28_) 2023-10-28 00:00:48,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__ns_testMinorCompaction__s98k0fae0gu5f3kgifkpkpopa2kh5136ark192bqc4d13siuk0r885ll8f9nv3cm/family1/6e2ad66f87514fb58203d27da5eec7e1 2023-10-28 00:00:48,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__ns_testMinorCompaction__20nmpgvadikm80aheegrqaa5bvrfdfnrcv5c6smc8l78pcp3fen3m3v2fmmqfplf/family1/2e4eebca101f4d54b2de8ad83ad9cf55 2023-10-28 00:00:48,072 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231028 table='ns:testMinorCompaction' column='family1' 2023-10-28 00:00:48,072 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231028 table='ns:testMinorCompaction' column='family1' 2023-10-28 00:00:48,072 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:48,072 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:48,073 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202310285338510605a34930ab701bdea8408e02 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202310285338510605a34930ab701bdea8408e02 2023-10-28 00:00:48,073 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-10-28 00:00:48,073 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202310280698adc416c74353b88291ba29af19fb to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202310280698adc416c74353b88291ba29af19fb 2023-10-28 00:00:48,073 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20231028' 2023-10-28 00:00:48,073 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-10-28 00:00:48,074 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20231028' 2023-10-28 00:00:48,074 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20231028 for table='ns:testMinorCompaction' and column='family1' 2023-10-28 00:00:48,074 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20231028 for table='ns:testMinorCompaction' and column='family1' 2023-10-28 00:00:48,411 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202310287e288f8cfc4d4b0aa504c3f2695eebe2 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202310287e288f8cfc4d4b0aa504c3f2695eebe2 2023-10-28 00:00:48,412 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231028 table='ns:testMinorCompaction' column='family1' 2023-10-28 00:00:48,439 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20231028/family1/b6d509472fd24a2498649ff15d52d839 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-10-28 00:00:48,440 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20231028/family1/b6d509472fd24a2498649ff15d52d839}] 2023-10-28 00:00:48,441 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20231028/family1/b6d509472fd24a2498649ff15d52d839 for inclusion in e6aa79e78506420304e660bae0e1ad9a/family1 2023-10-28 00:00:48,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-10-28 00:00:48,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(777): Region bounds: first=2 last= 2023-10-28 00:00:48,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for e6aa79e78506420304e660bae0e1ad9a: 2023-10-28 00:00:48,444 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20231028/family1/b6d509472fd24a2498649ff15d52d839 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__ns_testMinorCompaction__80uquorda302q2dnqbap21tr239m0crudks1ggnsjfoabmfs46n339vrn54ed1op/family1/b6d509472fd24a2498649ff15d52d839 2023-10-28 00:00:48,446 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__ns_testMinorCompaction__80uquorda302q2dnqbap21tr239m0crudks1ggnsjfoabmfs46n339vrn54ed1op/family1/b6d509472fd24a2498649ff15d52d839 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/062aa9450fc8429587eca09225c22d80_SeqId_28_ 2023-10-28 00:00:48,446 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__ns_testMinorCompaction__80uquorda302q2dnqbap21tr239m0crudks1ggnsjfoabmfs46n339vrn54ed1op/family1/b6d509472fd24a2498649ff15d52d839 into e6aa79e78506420304e660bae0e1ad9a/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/062aa9450fc8429587eca09225c22d80_SeqId_28_ - updating store file list. 2023-10-28 00:00:48,448 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/062aa9450fc8429587eca09225c22d80_SeqId_28_ into e6aa79e78506420304e660bae0e1ad9a/family1 2023-10-28 00:00:48,448 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__ns_testMinorCompaction__80uquorda302q2dnqbap21tr239m0crudks1ggnsjfoabmfs46n339vrn54ed1op/family1/b6d509472fd24a2498649ff15d52d839 into e6aa79e78506420304e660bae0e1ad9a/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/family1/062aa9450fc8429587eca09225c22d80_SeqId_28_) 2023-10-28 00:00:48,448 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__ns_testMinorCompaction__80uquorda302q2dnqbap21tr239m0crudks1ggnsjfoabmfs46n339vrn54ed1op/family1/b6d509472fd24a2498649ff15d52d839 2023-10-28 00:00:48,450 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231028 table='ns:testMinorCompaction' column='family1' 2023-10-28 00:00:48,450 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:00:48,451 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20231028b846b25b82bd4230b142100a15339a6d to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20231028b846b25b82bd4230b142100a15339a6d 2023-10-28 00:00:48,451 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-10-28 00:00:48,451 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20231028' 2023-10-28 00:00:48,452 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20231028 for table='ns:testMinorCompaction' and column='family1' 2023-10-28 00:00:48,452 INFO [Listener at localhost/40295] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-10-28 00:00:48,454 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-10-28 00:00:48,456 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-10-28 00:00:48,462 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-10-28 00:00:48,463 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-10-28 00:00:48,477 INFO [Listener at localhost/40295] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=249 (was 279), OpenFileDescriptor=632 (was 748), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=64 (was 82), ProcessCount=168 (was 168), AvailableMemoryMB=5906 (was 5914) 2023-10-28 00:00:48,489 INFO [Listener at localhost/40295] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=249, OpenFileDescriptor=632, MaxFileDescriptor=60000, SystemLoadAverage=64, ProcessCount=168, AvailableMemoryMB=5905 2023-10-28 00:00:48,491 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.HMaster$4(2112): Client=jenkins//188.40.62.62 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-10-28 00:00:48,492 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure2.ProcedureExecutor(1028): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:48,493 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-10-28 00:00:48,493 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(700): Client=jenkins//188.40.62.62 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-10-28 00:00:48,494 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=112 2023-10-28 00:00:48,494 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-10-28 00:00:48,496 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354 2023-10-28 00:00:48,496 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354 empty. 2023-10-28 00:00:48,496 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354 2023-10-28 00:00:48,496 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-10-28 00:00:48,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=112 2023-10-28 00:00:48,798 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=112 2023-10-28 00:00:48,906 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-10-28 00:00:48,909 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => f8230c1f516a2a5268355abb24416354, NAME => 'testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp 2023-10-28 00:00:49,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=112 2023-10-28 00:00:49,320 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-28 00:00:49,321 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing f8230c1f516a2a5268355abb24416354, disabling compactions & flushes 2023-10-28 00:00:49,321 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:00:49,321 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:00:49,321 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. after waiting 0 ms 2023-10-28 00:00:49,321 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:00:49,321 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:00:49,321 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for f8230c1f516a2a5268355abb24416354: 2023-10-28 00:00:49,326 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-10-28 00:00:49,328 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1698451249328"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451249328"}]},"ts":"1698451249328"} 2023-10-28 00:00:49,331 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-10-28 00:00:49,331 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-10-28 00:00:49,332 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451249331"}]},"ts":"1698451249331"} 2023-10-28 00:00:49,332 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-10-28 00:00:49,373 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=f8230c1f516a2a5268355abb24416354, ASSIGN}] 2023-10-28 00:00:49,376 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=f8230c1f516a2a5268355abb24416354, ASSIGN 2023-10-28 00:00:49,377 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=f8230c1f516a2a5268355abb24416354, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,43469,1698451096272; forceNewPlan=false, retain=false 2023-10-28 00:00:49,529 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=f8230c1f516a2a5268355abb24416354, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:49,530 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1698451249529"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451249529"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451249529"}]},"ts":"1698451249529"} 2023-10-28 00:00:49,533 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure f8230c1f516a2a5268355abb24416354, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-28 00:00:49,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=112 2023-10-28 00:00:49,688 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:49,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:00:49,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f8230c1f516a2a5268355abb24416354, NAME => 'testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354.', STARTKEY => '', ENDKEY => ''} 2023-10-28 00:00:49,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy f8230c1f516a2a5268355abb24416354 2023-10-28 00:00:49,696 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-28 00:00:49,696 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for f8230c1f516a2a5268355abb24416354 2023-10-28 00:00:49,696 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for f8230c1f516a2a5268355abb24416354 2023-10-28 00:00:49,699 INFO [StoreOpener-f8230c1f516a2a5268355abb24416354-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f8230c1f516a2a5268355abb24416354 2023-10-28 00:00:49,700 DEBUG [StoreOpener-f8230c1f516a2a5268355abb24416354-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1 2023-10-28 00:00:49,701 DEBUG [StoreOpener-f8230c1f516a2a5268355abb24416354-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1 2023-10-28 00:00:49,701 INFO [StoreOpener-f8230c1f516a2a5268355abb24416354-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f8230c1f516a2a5268355abb24416354 columnFamilyName family1 2023-10-28 00:00:49,702 INFO [StoreOpener-f8230c1f516a2a5268355abb24416354-1] regionserver.HStore(310): Store=f8230c1f516a2a5268355abb24416354/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-28 00:00:49,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354 2023-10-28 00:00:49,704 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354 2023-10-28 00:00:49,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for f8230c1f516a2a5268355abb24416354 2023-10-28 00:00:49,708 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-10-28 00:00:49,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened f8230c1f516a2a5268355abb24416354; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58763378, jitterRate=-0.1243574321269989}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-10-28 00:00:49,709 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for f8230c1f516a2a5268355abb24416354: 2023-10-28 00:00:49,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354., pid=114, masterSystemTime=1698451249688 2023-10-28 00:00:49,711 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:00:49,711 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:00:49,712 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=f8230c1f516a2a5268355abb24416354, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:49,712 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1698451249711"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451249711"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451249711"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451249711"}]},"ts":"1698451249711"} 2023-10-28 00:00:49,715 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=114, resume processing ppid=113 2023-10-28 00:00:49,715 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure f8230c1f516a2a5268355abb24416354, server=jenkins-hbase16.apache.org,43469,1698451096272 in 180 msec 2023-10-28 00:00:49,716 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=113, resume processing ppid=112 2023-10-28 00:00:49,716 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=f8230c1f516a2a5268355abb24416354, ASSIGN in 342 msec 2023-10-28 00:00:49,717 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-10-28 00:00:49,717 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451249717"}]},"ts":"1698451249717"} 2023-10-28 00:00:49,718 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-10-28 00:00:49,724 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-10-28 00:00:49,725 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 1.2330 sec 2023-10-28 00:00:50,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=112 2023-10-28 00:00:50,607 INFO [Listener at localhost/40295] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-10-28 00:00:50,635 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-28 00:00:50,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:50,637 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-10-28 00:00:50,637 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:50,637 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-10-28 00:00:50,637 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-10-28 00:00:50,637 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:50,637 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:50,673 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:50,673 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:50,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:50,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:50,674 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:50,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:50,674 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-28 00:00:50,674 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:50,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-28 00:00:50,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:50,674 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:50,674 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:50,674 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-10-28 00:00:50,674 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:50,674 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-10-28 00:00:50,675 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-28 00:00:50,675 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-28 00:00:50,675 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:00:50,675 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. started... 2023-10-28 00:00:50,675 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f8230c1f516a2a5268355abb24416354 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-10-28 00:00:51,100 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511309cb1346e939f49a882abda6bda6914fc to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511309cb1346e939f49a882abda6bda6914fc 2023-10-28 00:00:51,506 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/.tmp/family1/d9852f596cdf4260a6cc2e4cea7a5a8a 2023-10-28 00:00:51,527 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/.tmp/family1/d9852f596cdf4260a6cc2e4cea7a5a8a as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/d9852f596cdf4260a6cc2e4cea7a5a8a 2023-10-28 00:00:51,536 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/d9852f596cdf4260a6cc2e4cea7a5a8a, entries=1000, sequenceid=5, filesize=166.0 K 2023-10-28 00:00:51,537 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for f8230c1f516a2a5268355abb24416354 in 862ms, sequenceid=5, compaction requested=false 2023-10-28 00:00:51,537 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-10-28 00:00:51,538 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f8230c1f516a2a5268355abb24416354: 2023-10-28 00:00:51,538 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:00:51,538 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-28 00:00:51,538 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-28 00:00:51,538 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:51,538 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-10-28 00:00:51,538 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-10-28 00:00:51,558 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:51,558 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:51,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:51,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:51,559 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,559 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-28 00:00:51,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:51,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:51,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:51,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:51,562 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-10-28 00:00:51,563 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3d84b4b9[Count = 0] remaining members to acquire global barrier 2023-10-28 00:00:51,563 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-10-28 00:00:51,563 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,573 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,574 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-10-28 00:00:51,574 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-10-28 00:00:51,574 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-28 00:00:51,574 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:51,574 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-28 00:00:51,582 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-10-28 00:00:51,582 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:51,582 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:51,582 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-10-28 00:00:51,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:51,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:51,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:51,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:51,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:51,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:51,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:51,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:51,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-28 00:00:51,591 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-10-28 00:00:51,591 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-10-28 00:00:51,591 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-28 00:00:51,591 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-28 00:00:51,591 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,591 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-28 00:00:51,598 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,598 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,598 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:51,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:51,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:51,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,599 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:51,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:51,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:51,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:51,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:51,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,602 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:51,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:51,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:51,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:51,615 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:51,615 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:51,615 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,615 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:51,615 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,615 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:51,615 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,615 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,615 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,615 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-10-28 00:00:51,615 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-28 00:00:51,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:51,615 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:51,615 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-10-28 00:00:51,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:51,615 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-10-28 00:00:51,615 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-28 00:00:51,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:51,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:51,715 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-28 00:00:51,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-28 00:00:51,726 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-28 00:00:51,729 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-10-28 00:00:51,729 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,729 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-10-28 00:00:51,729 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:51,729 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-10-28 00:00:51,729 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-10-28 00:00:51,730 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,730 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,740 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:51,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:51,740 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:51,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:51,741 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:51,741 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-28 00:00:51,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,742 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-28 00:00:51,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,743 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,743 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-10-28 00:00:51,743 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:51,743 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-10-28 00:00:51,743 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:51,744 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-10-28 00:00:51,744 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-28 00:00:51,744 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-28 00:00:51,744 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:00:51,744 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. started... 2023-10-28 00:00:51,744 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f8230c1f516a2a5268355abb24416354 1/1 column families, dataSize=74 B heapSize=408 B 2023-10-28 00:00:52,168 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128e147d844a6cb4a998f384b845a63c4ab to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128e147d844a6cb4a998f384b845a63c4ab 2023-10-28 00:00:52,576 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/.tmp/family1/4503b81e45d9403d9bcce7a90190e92d 2023-10-28 00:00:52,584 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/.tmp/family1/4503b81e45d9403d9bcce7a90190e92d as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/4503b81e45d9403d9bcce7a90190e92d 2023-10-28 00:00:52,590 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/4503b81e45d9403d9bcce7a90190e92d, entries=1, sequenceid=9, filesize=5.0 K 2023-10-28 00:00:52,590 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for f8230c1f516a2a5268355abb24416354 in 846ms, sequenceid=9, compaction requested=false 2023-10-28 00:00:52,591 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f8230c1f516a2a5268355abb24416354: 2023-10-28 00:00:52,591 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:00:52,591 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-28 00:00:52,591 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-28 00:00:52,591 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:52,591 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-10-28 00:00:52,591 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-10-28 00:00:52,623 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,623 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:52,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:52,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:52,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:52,624 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,624 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-28 00:00:52,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:52,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:52,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:52,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:52,628 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-10-28 00:00:52,628 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@27c7ab4a[Count = 0] remaining members to acquire global barrier 2023-10-28 00:00:52,628 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-10-28 00:00:52,628 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,638 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,638 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-10-28 00:00:52,638 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-10-28 00:00:52,639 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-28 00:00:52,639 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:52,639 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-28 00:00:52,648 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-10-28 00:00:52,648 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:52,648 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:52,649 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-10-28 00:00:52,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:52,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:52,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:52,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:52,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:52,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:52,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:52,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:52,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-28 00:00:52,658 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-10-28 00:00:52,658 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-10-28 00:00:52,658 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-28 00:00:52,658 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-28 00:00:52,658 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,658 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-28 00:00:52,665 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,665 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,665 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:52,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:52,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:52,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,666 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:52,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:52,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:52,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:52,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:52,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,668 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:52,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:52,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:52,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:52,701 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:52,701 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:52,701 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:52,701 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-10-28 00:00:52,701 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-28 00:00:52,701 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,702 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,702 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:52,702 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,702 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,702 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,702 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-10-28 00:00:52,701 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:52,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:52,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:52,702 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-10-28 00:00:52,702 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-28 00:00:52,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:52,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:52,803 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-28 00:00:52,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-28 00:00:52,816 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-28 00:00:52,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-10-28 00:00:52,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,819 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-10-28 00:00:52,819 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:52,820 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-10-28 00:00:52,820 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-10-28 00:00:52,820 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,820 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,831 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:52,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:52,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:52,831 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:52,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,832 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:52,832 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-28 00:00:52,833 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-28 00:00:52,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,833 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,833 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-10-28 00:00:52,833 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:52,834 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-10-28 00:00:52,834 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:52,834 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-10-28 00:00:52,834 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-28 00:00:52,834 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-28 00:00:52,834 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:00:52,834 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. started... 2023-10-28 00:00:52,834 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f8230c1f516a2a5268355abb24416354 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-28 00:00:52,851 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201c61b859da1074932a865f1e059dd0bf1 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201c61b859da1074932a865f1e059dd0bf1 2023-10-28 00:00:53,259 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/.tmp/family1/577a31f9ff184294a6bfee99a47b44e9 2023-10-28 00:00:53,270 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/.tmp/family1/577a31f9ff184294a6bfee99a47b44e9 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/577a31f9ff184294a6bfee99a47b44e9 2023-10-28 00:00:53,277 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/577a31f9ff184294a6bfee99a47b44e9, entries=1, sequenceid=13, filesize=4.9 K 2023-10-28 00:00:53,278 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f8230c1f516a2a5268355abb24416354 in 444ms, sequenceid=13, compaction requested=true 2023-10-28 00:00:53,278 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f8230c1f516a2a5268355abb24416354: 2023-10-28 00:00:53,278 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:00:53,278 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-28 00:00:53,278 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-28 00:00:53,278 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:53,278 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-10-28 00:00:53,278 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-10-28 00:00:53,315 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,315 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:53,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:53,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:53,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:53,316 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,316 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-28 00:00:53,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:53,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:53,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:53,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:53,320 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-10-28 00:00:53,320 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@509d8b44[Count = 0] remaining members to acquire global barrier 2023-10-28 00:00:53,320 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-10-28 00:00:53,321 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,332 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,332 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-10-28 00:00:53,332 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-10-28 00:00:53,332 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-28 00:00:53,332 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:53,333 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-28 00:00:53,340 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:53,340 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-10-28 00:00:53,340 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:53,340 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-10-28 00:00:53,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:53,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:53,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:53,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:53,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:53,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:53,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:53,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:53,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-28 00:00:53,345 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-10-28 00:00:53,345 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-10-28 00:00:53,346 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-28 00:00:53,346 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-28 00:00:53,346 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,346 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-28 00:00:53,356 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,356 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:53,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:53,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,357 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:53,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:53,357 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:53,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:53,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:53,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:53,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:53,360 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:53,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:53,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:53,371 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:53,371 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,371 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,371 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:53,372 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-10-28 00:00:53,371 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:53,372 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:53,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:53,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:53,371 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-10-28 00:00:53,372 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-28 00:00:53,371 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:53,372 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,372 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,372 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:53,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:53,373 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-10-28 00:00:53,373 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-28 00:00:53,473 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-28 00:00:53,475 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-28 00:00:53,485 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-28 00:00:53,487 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-10-28 00:00:53,487 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,487 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-10-28 00:00:53,488 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:53,488 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-10-28 00:00:53,488 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-10-28 00:00:53,488 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,488 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,498 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:53,498 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:53,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:53,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:53,498 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:53,498 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-28 00:00:53,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,499 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-28 00:00:53,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,499 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,499 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-10-28 00:00:53,499 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,499 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-10-28 00:00:53,499 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:53,499 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-10-28 00:00:53,499 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-28 00:00:53,499 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-28 00:00:53,499 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:00:53,499 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. started... 2023-10-28 00:00:53,500 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f8230c1f516a2a5268355abb24416354 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-28 00:00:53,918 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205dd0830cb7e314b5db542d3c5d0571bde to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205dd0830cb7e314b5db542d3c5d0571bde 2023-10-28 00:00:53,925 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/.tmp/family1/d19912f284a64c888bcd09aad43ce5f5 2023-10-28 00:00:53,931 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/.tmp/family1/d19912f284a64c888bcd09aad43ce5f5 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/d19912f284a64c888bcd09aad43ce5f5 2023-10-28 00:00:53,938 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/d19912f284a64c888bcd09aad43ce5f5, entries=1, sequenceid=17, filesize=4.9 K 2023-10-28 00:00:53,939 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f8230c1f516a2a5268355abb24416354 in 439ms, sequenceid=17, compaction requested=true 2023-10-28 00:00:53,939 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f8230c1f516a2a5268355abb24416354: 2023-10-28 00:00:53,939 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:00:53,939 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-28 00:00:53,939 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-28 00:00:53,939 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:53,939 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-10-28 00:00:53,939 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-10-28 00:00:53,973 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:53,973 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:53,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:53,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:53,974 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,974 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-28 00:00:53,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:53,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:53,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:53,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:53,976 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-10-28 00:00:53,976 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2f777ca0[Count = 0] remaining members to acquire global barrier 2023-10-28 00:00:53,976 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-10-28 00:00:53,976 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,981 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,981 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-10-28 00:00:53,982 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-10-28 00:00:53,982 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-28 00:00:53,982 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:53,982 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-28 00:00:53,990 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-10-28 00:00:53,990 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:53,990 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:53,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:53,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:53,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:53,990 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-10-28 00:00:53,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:53,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:53,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:53,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:53,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:53,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-28 00:00:53,996 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-10-28 00:00:53,996 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-10-28 00:00:53,996 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-28 00:00:53,996 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-28 00:00:53,996 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:53,996 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-28 00:00:54,006 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:54,006 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:54,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:54,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:54,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:54,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:54,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:54,007 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:54,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:54,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:54,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:54,008 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:54,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:54,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:54,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:54,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:54,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:54,011 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:54,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:54,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:54,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:54,023 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:54,023 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:54,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:54,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:54,023 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-10-28 00:00:54,023 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-28 00:00:54,023 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:54,023 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:54,023 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:54,024 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:54,024 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-10-28 00:00:54,024 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-28 00:00:54,024 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:54,024 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:54,024 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:54,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:54,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:54,024 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:54,124 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-28 00:00:54,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-28 00:00:54,135 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-28 00:00:54,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-10-28 00:00:54,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:54,139 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-10-28 00:00:54,139 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:54,140 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-10-28 00:00:54,140 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-10-28 00:00:54,141 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:54,141 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:54,148 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:54,148 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:54,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:54,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:54,149 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:54,149 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-28 00:00:54,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:54,150 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:54,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-28 00:00:54,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:54,152 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:54,152 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-10-28 00:00:54,152 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:54,153 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-10-28 00:00:54,153 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:54,153 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-10-28 00:00:54,153 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-28 00:00:54,153 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-28 00:00:54,153 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:00:54,154 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. started... 2023-10-28 00:00:54,154 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f8230c1f516a2a5268355abb24416354 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-28 00:00:54,580 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228f5313d09c46a42db838acdd94c59bdcc to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228f5313d09c46a42db838acdd94c59bdcc 2023-10-28 00:00:54,987 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/.tmp/family1/b59996b5cdfc4c14a39e6854396a729c 2023-10-28 00:00:54,999 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/.tmp/family1/b59996b5cdfc4c14a39e6854396a729c as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/b59996b5cdfc4c14a39e6854396a729c 2023-10-28 00:00:55,003 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/b59996b5cdfc4c14a39e6854396a729c, entries=1, sequenceid=21, filesize=4.9 K 2023-10-28 00:00:55,004 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f8230c1f516a2a5268355abb24416354 in 850ms, sequenceid=21, compaction requested=true 2023-10-28 00:00:55,004 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f8230c1f516a2a5268355abb24416354: 2023-10-28 00:00:55,004 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:00:55,004 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-28 00:00:55,004 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-28 00:00:55,004 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,004 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-10-28 00:00:55,004 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-10-28 00:00:55,015 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,015 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:55,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:55,015 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,015 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-28 00:00:55,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:55,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:55,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:55,018 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-10-28 00:00:55,018 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@37ce248b[Count = 0] remaining members to acquire global barrier 2023-10-28 00:00:55,018 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-10-28 00:00:55,019 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,023 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,023 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-10-28 00:00:55,023 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-10-28 00:00:55,023 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-28 00:00:55,024 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,024 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-28 00:00:55,031 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-10-28 00:00:55,031 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:55,032 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-10-28 00:00:55,031 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:55,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:55,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:55,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:55,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:55,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-28 00:00:55,038 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-10-28 00:00:55,038 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-10-28 00:00:55,039 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-28 00:00:55,039 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-28 00:00:55,039 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,039 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-28 00:00:55,076 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,077 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,077 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:55,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:55,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:55,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:55,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:55,079 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:55,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:55,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,083 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:55,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,105 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,105 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,105 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,105 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,105 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:55,105 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:55,105 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-10-28 00:00:55,105 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-28 00:00:55,105 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,106 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,106 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,105 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:55,106 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-10-28 00:00:55,106 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-10-28 00:00:55,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:55,106 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-28 00:00:55,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:55,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:55,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:55,206 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-28 00:00:55,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-28 00:00:55,218 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-28 00:00:55,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-10-28 00:00:55,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,221 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-10-28 00:00:55,221 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:55,221 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-10-28 00:00:55,221 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-10-28 00:00:55,222 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,222 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,230 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:55,230 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:55,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:55,231 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,231 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-28 00:00:55,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,232 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-28 00:00:55,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,233 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,233 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-10-28 00:00:55,233 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-10-28 00:00:55,233 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,233 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-10-28 00:00:55,234 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:55,234 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-10-28 00:00:55,234 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-28 00:00:55,234 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-28 00:00:55,234 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:00:55,234 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. started... 2023-10-28 00:00:55,234 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f8230c1f516a2a5268355abb24416354 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-28 00:00:55,658 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512317943d2cd4815493aa80bed185a6ce7c9 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512317943d2cd4815493aa80bed185a6ce7c9 2023-10-28 00:00:55,665 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/.tmp/family1/9fc617f953334df48abeecee8d31c73e 2023-10-28 00:00:55,672 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/.tmp/family1/9fc617f953334df48abeecee8d31c73e as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/9fc617f953334df48abeecee8d31c73e 2023-10-28 00:00:55,677 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/9fc617f953334df48abeecee8d31c73e, entries=1, sequenceid=25, filesize=4.9 K 2023-10-28 00:00:55,678 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f8230c1f516a2a5268355abb24416354 in 444ms, sequenceid=25, compaction requested=true 2023-10-28 00:00:55,679 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f8230c1f516a2a5268355abb24416354: 2023-10-28 00:00:55,679 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:00:55,679 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-28 00:00:55,679 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-28 00:00:55,679 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,679 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-10-28 00:00:55,679 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-10-28 00:00:55,713 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,713 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:55,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:55,714 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,714 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-28 00:00:55,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:55,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:55,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:55,716 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-10-28 00:00:55,716 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2d4954c3[Count = 0] remaining members to acquire global barrier 2023-10-28 00:00:55,716 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-10-28 00:00:55,716 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,721 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,721 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-10-28 00:00:55,721 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-10-28 00:00:55,721 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-28 00:00:55,722 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,722 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-28 00:00:55,730 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-10-28 00:00:55,730 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,730 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:55,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:55,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:55,730 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-10-28 00:00:55,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:55,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:55,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:55,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-28 00:00:55,734 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-10-28 00:00:55,734 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-10-28 00:00:55,735 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-28 00:00:55,735 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-28 00:00:55,735 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,735 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-28 00:00:55,746 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,746 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:55,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:55,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,747 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:55,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:55,747 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:55,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:55,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:55,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,750 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:55,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,763 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,763 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:55,763 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:55,763 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,764 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,764 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,764 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,764 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,764 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,763 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-10-28 00:00:55,764 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-28 00:00:55,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:55,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:55,763 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:55,764 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-10-28 00:00:55,764 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-10-28 00:00:55,765 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-28 00:00:55,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:55,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:55,865 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-28 00:00:55,867 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-28 00:00:55,876 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-28 00:00:55,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-10-28 00:00:55,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,879 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-10-28 00:00:55,879 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:55,879 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-10-28 00:00:55,879 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-10-28 00:00:55,879 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,879 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,888 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:55,888 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:55,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:55,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,888 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:55,888 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-28 00:00:55,888 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-28 00:00:55,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,889 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,889 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-10-28 00:00:55,889 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:55,889 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-10-28 00:00:55,889 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:55,889 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-10-28 00:00:55,889 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-28 00:00:55,889 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-28 00:00:55,889 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:00:55,889 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. started... 2023-10-28 00:00:55,890 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f8230c1f516a2a5268355abb24416354 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-28 00:00:56,301 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101384bdc4ae056408ea0e16a40bbbc647f to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101384bdc4ae056408ea0e16a40bbbc647f 2023-10-28 00:00:56,706 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/.tmp/family1/0d0d75d72a434a3c832b499cea44c31b 2023-10-28 00:00:56,718 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/.tmp/family1/0d0d75d72a434a3c832b499cea44c31b as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/0d0d75d72a434a3c832b499cea44c31b 2023-10-28 00:00:56,723 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/0d0d75d72a434a3c832b499cea44c31b, entries=1, sequenceid=29, filesize=4.9 K 2023-10-28 00:00:56,723 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f8230c1f516a2a5268355abb24416354 in 833ms, sequenceid=29, compaction requested=true 2023-10-28 00:00:56,724 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f8230c1f516a2a5268355abb24416354: 2023-10-28 00:00:56,724 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:00:56,724 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-28 00:00:56,724 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-28 00:00:56,724 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:56,724 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-10-28 00:00:56,724 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-10-28 00:00:56,761 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:56,761 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:56,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:56,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:56,762 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,762 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-28 00:00:56,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:56,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:56,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:56,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:56,766 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-10-28 00:00:56,766 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@11b22d85[Count = 0] remaining members to acquire global barrier 2023-10-28 00:00:56,766 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-10-28 00:00:56,766 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,771 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,772 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-10-28 00:00:56,772 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-10-28 00:00:56,772 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-28 00:00:56,772 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:56,772 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-28 00:00:56,780 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-10-28 00:00:56,780 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:56,780 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-10-28 00:00:56,780 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:56,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:56,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:56,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:56,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:56,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:56,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:56,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:56,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:56,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-28 00:00:56,788 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-10-28 00:00:56,788 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-10-28 00:00:56,788 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-28 00:00:56,788 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-28 00:00:56,788 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,788 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-28 00:00:56,796 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,797 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:56,796 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:56,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:56,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:56,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:56,798 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:56,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:56,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:56,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,801 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:56,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:56,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:56,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:56,813 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:56,813 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,813 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-10-28 00:00:56,813 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-28 00:00:56,813 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:56,814 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:56,813 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,813 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:56,814 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-10-28 00:00:56,814 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-28 00:00:56,814 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:56,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:56,814 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,814 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,814 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,814 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-10-28 00:00:56,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:56,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:56,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:56,914 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-28 00:00:56,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-28 00:00:56,926 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-28 00:00:56,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-10-28 00:00:56,928 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,928 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-10-28 00:00:56,928 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:56,928 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-10-28 00:00:56,928 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-10-28 00:00:56,928 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,928 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,938 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:56,938 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:56,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:56,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:56,938 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:56,938 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-28 00:00:56,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,939 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-28 00:00:56,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,940 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,940 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-10-28 00:00:56,940 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:56,940 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-10-28 00:00:56,940 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:56,940 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-10-28 00:00:56,940 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-28 00:00:56,940 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-28 00:00:56,940 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:00:56,940 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. started... 2023-10-28 00:00:56,940 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f8230c1f516a2a5268355abb24416354 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-28 00:00:57,360 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010380beb1c26ee347f78a676fa4d80c6bee to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010380beb1c26ee347f78a676fa4d80c6bee 2023-10-28 00:00:57,767 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/.tmp/family1/5267e822a643458fa4bf0876ac84d00e 2023-10-28 00:00:57,777 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/.tmp/family1/5267e822a643458fa4bf0876ac84d00e as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/5267e822a643458fa4bf0876ac84d00e 2023-10-28 00:00:57,784 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/5267e822a643458fa4bf0876ac84d00e, entries=1, sequenceid=33, filesize=4.9 K 2023-10-28 00:00:57,785 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f8230c1f516a2a5268355abb24416354 in 845ms, sequenceid=33, compaction requested=true 2023-10-28 00:00:57,785 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f8230c1f516a2a5268355abb24416354: 2023-10-28 00:00:57,785 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:00:57,785 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-28 00:00:57,785 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-28 00:00:57,786 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:57,786 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-10-28 00:00:57,786 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-10-28 00:00:57,819 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:57,820 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:57,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:57,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:57,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:57,820 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:57,820 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-28 00:00:57,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:57,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:57,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:57,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:57,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:57,825 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-10-28 00:00:57,825 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5b4d16f[Count = 0] remaining members to acquire global barrier 2023-10-28 00:00:57,825 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-10-28 00:00:57,825 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:57,836 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:57,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:57,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:57,837 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-10-28 00:00:57,837 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-10-28 00:00:57,837 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-28 00:00:57,837 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:57,837 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-28 00:00:57,863 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-10-28 00:00:57,863 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:57,863 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:57,863 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-10-28 00:00:57,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:57,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:57,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:57,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:57,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:57,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:57,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:57,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:57,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:57,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:57,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-28 00:00:57,874 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-10-28 00:00:57,874 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-10-28 00:00:57,874 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-28 00:00:57,874 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-28 00:00:57,874 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:57,874 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-28 00:00:57,888 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:57,888 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:57,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:57,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:57,888 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:57,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:57,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:57,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:57,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:57,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:57,889 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:57,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:57,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:57,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:57,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:57,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:57,891 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:57,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:57,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:57,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:57,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:57,903 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:57,903 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:57,903 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:57,903 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-10-28 00:00:57,903 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-28 00:00:57,903 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:57,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:57,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:57,903 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:57,904 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-10-28 00:00:57,903 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:57,904 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-10-28 00:00:57,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:57,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:57,903 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:57,904 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-28 00:00:57,905 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:57,905 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:57,905 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,005 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-28 00:00:58,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-28 00:00:58,009 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-28 00:00:58,010 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-10-28 00:00:58,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,011 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-10-28 00:00:58,011 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:58,011 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-10-28 00:00:58,011 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-10-28 00:00:58,011 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,011 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,021 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:58,021 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:58,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:58,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:58,022 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:58,022 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-28 00:00:58,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,022 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-28 00:00:58,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,023 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,023 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-10-28 00:00:58,023 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,023 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-10-28 00:00:58,023 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:58,023 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-10-28 00:00:58,023 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-28 00:00:58,023 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-28 00:00:58,023 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:00:58,023 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. started... 2023-10-28 00:00:58,023 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f8230c1f516a2a5268355abb24416354 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-28 00:00:58,436 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907f0863a733a4a49979f94e0ab70646150 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20150907f0863a733a4a49979f94e0ab70646150 2023-10-28 00:00:58,845 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/.tmp/family1/f9ca364925c441268deda0e5ddcfbe03 2023-10-28 00:00:58,858 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/.tmp/family1/f9ca364925c441268deda0e5ddcfbe03 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/f9ca364925c441268deda0e5ddcfbe03 2023-10-28 00:00:58,863 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/f9ca364925c441268deda0e5ddcfbe03, entries=1, sequenceid=37, filesize=4.9 K 2023-10-28 00:00:58,864 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f8230c1f516a2a5268355abb24416354 in 841ms, sequenceid=37, compaction requested=true 2023-10-28 00:00:58,864 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f8230c1f516a2a5268355abb24416354: 2023-10-28 00:00:58,864 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:00:58,864 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-28 00:00:58,864 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-28 00:00:58,864 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:58,865 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-10-28 00:00:58,865 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-10-28 00:00:58,896 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:58,896 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:58,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:58,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:58,897 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,897 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-28 00:00:58,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:58,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:58,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:58,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:58,900 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-10-28 00:00:58,900 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4398472e[Count = 0] remaining members to acquire global barrier 2023-10-28 00:00:58,900 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-10-28 00:00:58,900 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,905 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,905 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-10-28 00:00:58,905 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-10-28 00:00:58,905 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-28 00:00:58,905 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:58,905 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-28 00:00:58,913 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:58,913 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-10-28 00:00:58,913 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:58,913 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-10-28 00:00:58,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:58,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:58,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:58,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:58,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:58,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:58,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:58,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:58,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-28 00:00:58,921 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-10-28 00:00:58,921 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-10-28 00:00:58,922 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-28 00:00:58,922 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-28 00:00:58,922 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,922 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-28 00:00:58,930 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,930 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,930 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:58,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:58,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:58,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:58,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:58,931 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:58,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:58,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:58,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,934 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:58,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:58,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:58,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:58,946 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:58,946 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:58,946 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:58,946 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,947 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,947 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:58,947 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,947 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,947 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:58,946 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-10-28 00:00:58,947 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-28 00:00:58,947 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-10-28 00:00:58,946 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:58,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:58,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:58,947 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-10-28 00:00:58,947 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-28 00:00:58,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:58,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:59,048 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-28 00:00:59,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-28 00:00:59,054 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-28 00:00:59,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-10-28 00:00:59,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:59,057 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-10-28 00:00:59,057 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:59,057 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-10-28 00:00:59,057 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-10-28 00:00:59,058 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:59,058 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:59,069 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:00:59,070 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:59,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:00:59,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:00:59,071 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:59,071 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-28 00:00:59,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:59,072 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:59,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-28 00:00:59,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:59,073 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:59,074 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-10-28 00:00:59,074 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:59,074 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-10-28 00:00:59,074 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:00:59,074 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-10-28 00:00:59,075 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-10-28 00:00:59,075 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-28 00:00:59,075 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:00:59,075 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. started... 2023-10-28 00:00:59,075 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f8230c1f516a2a5268355abb24416354 1/1 column families, dataSize=73 B heapSize=408 B 2023-10-28 00:00:59,500 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511205eea7cb6af214277809843d8d7de1e15 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511205eea7cb6af214277809843d8d7de1e15 2023-10-28 00:00:59,907 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/.tmp/family1/758a395e2b9941949c920b59f74acd2b 2023-10-28 00:00:59,921 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/.tmp/family1/758a395e2b9941949c920b59f74acd2b as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/758a395e2b9941949c920b59f74acd2b 2023-10-28 00:00:59,930 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/758a395e2b9941949c920b59f74acd2b, entries=1, sequenceid=41, filesize=4.9 K 2023-10-28 00:00:59,931 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f8230c1f516a2a5268355abb24416354 in 856ms, sequenceid=41, compaction requested=true 2023-10-28 00:00:59,931 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f8230c1f516a2a5268355abb24416354: 2023-10-28 00:00:59,931 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:00:59,931 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-10-28 00:00:59,932 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-10-28 00:00:59,932 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:59,932 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-10-28 00:00:59,932 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-10-28 00:00:59,954 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:59,955 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:59,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:59,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:59,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:59,955 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:59,955 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-28 00:00:59,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:59,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:59,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:59,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:59,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:59,959 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-10-28 00:00:59,959 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2490fcd2[Count = 0] remaining members to acquire global barrier 2023-10-28 00:00:59,959 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-10-28 00:00:59,959 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:59,969 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:59,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:59,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:59,970 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-10-28 00:00:59,970 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-10-28 00:00:59,970 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-28 00:00:59,970 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:59,970 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-28 00:00:59,980 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:59,980 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-10-28 00:00:59,980 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:00:59,980 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-10-28 00:00:59,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:59,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:59,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:59,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:59,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:00:59,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:59,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:59,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:00:59,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:59,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:59,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-28 00:00:59,988 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-10-28 00:00:59,988 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-10-28 00:00:59,988 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-28 00:00:59,989 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-28 00:00:59,989 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:59,989 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-28 00:00:59,996 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:59,996 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:59,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:59,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:00:59,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:00:59,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:59,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:59,997 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:00:59,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:00:59,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:00:59,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:00:59,998 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:00:59,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:00:59,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:01:00,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:01:00,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:01:00,001 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:00,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:00,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:01:00,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-10-28 00:01:00,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:00,013 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:00,013 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:01:00,013 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:01:00,013 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-10-28 00:01:00,013 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-28 00:01:00,013 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-10-28 00:01:00,013 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:01:00,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:01:00,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:01:00,013 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:01:00,013 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:01:00,014 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-10-28 00:01:00,014 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-28 00:01:00,014 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:00,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:01:00,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:01:00,015 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:01:00,015 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:01:00,015 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-10-28 00:01:00,115 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-28 00:01:00,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-28 00:01:00,121 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36413] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-10-28 00:01:00,124 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] procedure2.ProcedureExecutor(1028): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-10-28 00:01:00,126 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-10-28 00:01:00,126 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698451860126 2023-10-28 00:01:00,126 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-28 00:01:00,131 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-10-28 00:01:00,132 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-10-28 00:01:00,132 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-10-28 00:01:00,133 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] zookeeper.ReadOnlyZKClient(139): Connect 0x4c6faa71 to 127.0.0.1:57369 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-10-28 00:01:00,145 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4e94da19, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-28 00:01:00,145 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-10-28 00:01:00,989 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601039baf66756ce94fdaaffe4fa3aed051dd to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601039baf66756ce94fdaaffe4fa3aed051dd 2023-10-28 00:01:00,990 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-10-28 00:01:01,004 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-10-28 00:01:01,006 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:49044, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-10-28 00:01:01,012 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-10-28 00:01:01,013 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:44108, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-10-28 00:01:01,019 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/d9562f0cbf454dd5885ad4104755f6a3 first=Optional[r5] last=Optional[r6] 2023-10-28 00:01:01,020 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/d9562f0cbf454dd5885ad4104755f6a3}] 2023-10-28 00:01:01,027 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/d9562f0cbf454dd5885ad4104755f6a3 for inclusion in f8230c1f516a2a5268355abb24416354/family1 2023-10-28 00:01:01,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-10-28 00:01:01,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(777): Region bounds: first= last= 2023-10-28 00:01:01,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for f8230c1f516a2a5268355abb24416354: 2023-10-28 00:01:01,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/d9562f0cbf454dd5885ad4104755f6a3 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMinorCompactionWithMonthlyPolicy__st0v3j6p7cppdp37ua39730jlk143tkfo3nombar814n4q4nm1caq606g43nbbsn/family1/d9562f0cbf454dd5885ad4104755f6a3 2023-10-28 00:01:01,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMinorCompactionWithMonthlyPolicy__st0v3j6p7cppdp37ua39730jlk143tkfo3nombar814n4q4nm1caq606g43nbbsn/family1/d9562f0cbf454dd5885ad4104755f6a3 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/434f9b982899422f898dec51e5fc4a67_SeqId_44_ 2023-10-28 00:01:01,033 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMinorCompactionWithMonthlyPolicy__st0v3j6p7cppdp37ua39730jlk143tkfo3nombar814n4q4nm1caq606g43nbbsn/family1/d9562f0cbf454dd5885ad4104755f6a3 into f8230c1f516a2a5268355abb24416354/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/434f9b982899422f898dec51e5fc4a67_SeqId_44_ - updating store file list. 2023-10-28 00:01:01,038 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/434f9b982899422f898dec51e5fc4a67_SeqId_44_ into f8230c1f516a2a5268355abb24416354/family1 2023-10-28 00:01:01,038 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMinorCompactionWithMonthlyPolicy__st0v3j6p7cppdp37ua39730jlk143tkfo3nombar814n4q4nm1caq606g43nbbsn/family1/d9562f0cbf454dd5885ad4104755f6a3 into f8230c1f516a2a5268355abb24416354/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/434f9b982899422f898dec51e5fc4a67_SeqId_44_) 2023-10-28 00:01:01,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMinorCompactionWithMonthlyPolicy__st0v3j6p7cppdp37ua39730jlk143tkfo3nombar814n4q4nm1caq606g43nbbsn/family1/d9562f0cbf454dd5885ad4104755f6a3 2023-10-28 00:01:01,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-10-28 00:01:01,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-10-28 00:01:01,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(1912): f8230c1f516a2a5268355abb24416354/family1 is initiating minor compaction 2023-10-28 00:01:01,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@5dbbd915; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-28 00:01:01,042 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of f8230c1f516a2a5268355abb24416354/family1 in testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:01:01,042 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/4503b81e45d9403d9bcce7a90190e92d, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/577a31f9ff184294a6bfee99a47b44e9, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/d19912f284a64c888bcd09aad43ce5f5, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/b59996b5cdfc4c14a39e6854396a729c, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/9fc617f953334df48abeecee8d31c73e, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/0d0d75d72a434a3c832b499cea44c31b, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/5267e822a643458fa4bf0876ac84d00e, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/f9ca364925c441268deda0e5ddcfbe03, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/758a395e2b9941949c920b59f74acd2b, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/434f9b982899422f898dec51e5fc4a67_SeqId_44_] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/.tmp, totalSize=49.7 K 2023-10-28 00:01:01,042 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 4503b81e45d9403d9bcce7a90190e92d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-10-28 00:01:01,042 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 577a31f9ff184294a6bfee99a47b44e9, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-10-28 00:01:01,042 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting d19912f284a64c888bcd09aad43ce5f5, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-10-28 00:01:01,042 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting b59996b5cdfc4c14a39e6854396a729c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-10-28 00:01:01,042 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 9fc617f953334df48abeecee8d31c73e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-10-28 00:01:01,043 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 0d0d75d72a434a3c832b499cea44c31b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-10-28 00:01:01,043 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 5267e822a643458fa4bf0876ac84d00e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-10-28 00:01:01,043 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting f9ca364925c441268deda0e5ddcfbe03, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-10-28 00:01:01,043 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 758a395e2b9941949c920b59f74acd2b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-10-28 00:01:01,043 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 434f9b982899422f898dec51e5fc4a67_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-10-28 00:01:01,044 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-10-28 00:01:01,044 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:01:01,045 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101384bdc4ae056408ea0e16a40bbbc647f to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101384bdc4ae056408ea0e16a40bbbc647f 2023-10-28 00:01:01,046 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010380beb1c26ee347f78a676fa4d80c6bee to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010380beb1c26ee347f78a676fa4d80c6bee 2023-10-28 00:01:01,046 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-10-28 00:01:01,046 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-10-28 00:01:01,047 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-10-28 00:01:01,047 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-10-28 00:01:01,074 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] throttle.PressureAwareThroughputController(145): f8230c1f516a2a5268355abb24416354#family1#compaction#166 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 57.69 MB/second 2023-10-28 00:01:01,099 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123156ea3dc63e9c467d827241d346eb335a to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015123156ea3dc63e9c467d827241d346eb335a 2023-10-28 00:01:01,100 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-10-28 00:01:01,120 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/84e344bc598e49f2b09597e41371f50d first=Optional[r1] last=Optional[r4] 2023-10-28 00:01:01,120 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/84e344bc598e49f2b09597e41371f50d}] 2023-10-28 00:01:01,122 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/84e344bc598e49f2b09597e41371f50d for inclusion in f8230c1f516a2a5268355abb24416354/family1 2023-10-28 00:01:01,124 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-10-28 00:01:01,124 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(777): Region bounds: first= last= 2023-10-28 00:01:01,124 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for f8230c1f516a2a5268355abb24416354: 2023-10-28 00:01:01,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/84e344bc598e49f2b09597e41371f50d to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMinorCompactionWithMonthlyPolicy__2mv7mle3m5hfmctrqgu7tlsn0hvior9lr5g8hamjb1d3neh4o638gvbpih6kf9j3/family1/84e344bc598e49f2b09597e41371f50d 2023-10-28 00:01:01,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMinorCompactionWithMonthlyPolicy__2mv7mle3m5hfmctrqgu7tlsn0hvior9lr5g8hamjb1d3neh4o638gvbpih6kf9j3/family1/84e344bc598e49f2b09597e41371f50d as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/ebf2556d4dfb492591fbd6cb65b54c7c_SeqId_46_ 2023-10-28 00:01:01,127 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMinorCompactionWithMonthlyPolicy__2mv7mle3m5hfmctrqgu7tlsn0hvior9lr5g8hamjb1d3neh4o638gvbpih6kf9j3/family1/84e344bc598e49f2b09597e41371f50d into f8230c1f516a2a5268355abb24416354/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/ebf2556d4dfb492591fbd6cb65b54c7c_SeqId_46_ - updating store file list. 2023-10-28 00:01:01,130 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/ebf2556d4dfb492591fbd6cb65b54c7c_SeqId_46_ into f8230c1f516a2a5268355abb24416354/family1 2023-10-28 00:01:01,130 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMinorCompactionWithMonthlyPolicy__2mv7mle3m5hfmctrqgu7tlsn0hvior9lr5g8hamjb1d3neh4o638gvbpih6kf9j3/family1/84e344bc598e49f2b09597e41371f50d into f8230c1f516a2a5268355abb24416354/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/ebf2556d4dfb492591fbd6cb65b54c7c_SeqId_46_) 2023-10-28 00:01:01,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMinorCompactionWithMonthlyPolicy__2mv7mle3m5hfmctrqgu7tlsn0hvior9lr5g8hamjb1d3neh4o638gvbpih6kf9j3/family1/84e344bc598e49f2b09597e41371f50d 2023-10-28 00:01:01,132 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-10-28 00:01:01,132 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:01:01,133 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201c61b859da1074932a865f1e059dd0bf1 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201c61b859da1074932a865f1e059dd0bf1 2023-10-28 00:01:01,134 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205dd0830cb7e314b5db542d3c5d0571bde to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205dd0830cb7e314b5db542d3c5d0571bde 2023-10-28 00:01:01,135 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228f5313d09c46a42db838acdd94c59bdcc to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228f5313d09c46a42db838acdd94c59bdcc 2023-10-28 00:01:01,136 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512317943d2cd4815493aa80bed185a6ce7c9 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512317943d2cd4815493aa80bed185a6ce7c9 2023-10-28 00:01:01,136 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-10-28 00:01:01,136 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-10-28 00:01:01,136 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-10-28 00:01:01,137 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-10-28 00:01:01,503 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/.tmp/family1/84e9e6efba454b689c20f22510e2f783 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/84e9e6efba454b689c20f22510e2f783 2023-10-28 00:01:01,507 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in f8230c1f516a2a5268355abb24416354/family1 of f8230c1f516a2a5268355abb24416354 into 84e9e6efba454b689c20f22510e2f783(size=6.6 K), total size for store is 178.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-10-28 00:01:01,507 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for f8230c1f516a2a5268355abb24416354: 2023-10-28 00:01:01,507 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1698451261042; duration=0sec 2023-10-28 00:01:01,507 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-28 00:01:01,529 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/4503b81e45d9403d9bcce7a90190e92d, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/577a31f9ff184294a6bfee99a47b44e9, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/d19912f284a64c888bcd09aad43ce5f5, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/b59996b5cdfc4c14a39e6854396a729c, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/9fc617f953334df48abeecee8d31c73e, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/0d0d75d72a434a3c832b499cea44c31b, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/5267e822a643458fa4bf0876ac84d00e, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/f9ca364925c441268deda0e5ddcfbe03, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/758a395e2b9941949c920b59f74acd2b, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/434f9b982899422f898dec51e5fc4a67_SeqId_44_] to archive 2023-10-28 00:01:01,530 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:01:01,532 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/4503b81e45d9403d9bcce7a90190e92d to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/4503b81e45d9403d9bcce7a90190e92d 2023-10-28 00:01:01,533 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/577a31f9ff184294a6bfee99a47b44e9 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/577a31f9ff184294a6bfee99a47b44e9 2023-10-28 00:01:01,535 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/d19912f284a64c888bcd09aad43ce5f5 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/d19912f284a64c888bcd09aad43ce5f5 2023-10-28 00:01:01,536 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/b59996b5cdfc4c14a39e6854396a729c to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/b59996b5cdfc4c14a39e6854396a729c 2023-10-28 00:01:01,537 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/9fc617f953334df48abeecee8d31c73e to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/9fc617f953334df48abeecee8d31c73e 2023-10-28 00:01:01,538 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/0d0d75d72a434a3c832b499cea44c31b to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/0d0d75d72a434a3c832b499cea44c31b 2023-10-28 00:01:01,539 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/5267e822a643458fa4bf0876ac84d00e to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/5267e822a643458fa4bf0876ac84d00e 2023-10-28 00:01:01,541 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/f9ca364925c441268deda0e5ddcfbe03 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/f9ca364925c441268deda0e5ddcfbe03 2023-10-28 00:01:01,542 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/758a395e2b9941949c920b59f74acd2b to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/758a395e2b9941949c920b59f74acd2b 2023-10-28 00:01:01,543 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/434f9b982899422f898dec51e5fc4a67_SeqId_44_ to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/434f9b982899422f898dec51e5fc4a67_SeqId_44_ 2023-10-28 00:01:01,973 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130367ff501e0c6474eafb500d725639c2e to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130367ff501e0c6474eafb500d725639c2e 2023-10-28 00:01:01,974 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-10-28 00:01:01,998 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/883a123cebb849e0bf8b64966598a09c first=Optional[r00] last=Optional[r8] 2023-10-28 00:01:01,999 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/883a123cebb849e0bf8b64966598a09c}] 2023-10-28 00:01:02,001 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/883a123cebb849e0bf8b64966598a09c for inclusion in f8230c1f516a2a5268355abb24416354/family1 2023-10-28 00:01:02,005 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-10-28 00:01:02,005 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(777): Region bounds: first= last= 2023-10-28 00:01:02,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for f8230c1f516a2a5268355abb24416354: 2023-10-28 00:01:02,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/883a123cebb849e0bf8b64966598a09c to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMinorCompactionWithMonthlyPolicy__r8sbl93v408q2v4c281vuab0p19utck0qjrnalf51kdq2ikv7ilcjbk2k9g89i57/family1/883a123cebb849e0bf8b64966598a09c 2023-10-28 00:01:02,008 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMinorCompactionWithMonthlyPolicy__r8sbl93v408q2v4c281vuab0p19utck0qjrnalf51kdq2ikv7ilcjbk2k9g89i57/family1/883a123cebb849e0bf8b64966598a09c as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/6c729f49e6ad4e8abeab467620b66144_SeqId_49_ 2023-10-28 00:01:02,009 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMinorCompactionWithMonthlyPolicy__r8sbl93v408q2v4c281vuab0p19utck0qjrnalf51kdq2ikv7ilcjbk2k9g89i57/family1/883a123cebb849e0bf8b64966598a09c into f8230c1f516a2a5268355abb24416354/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/6c729f49e6ad4e8abeab467620b66144_SeqId_49_ - updating store file list. 2023-10-28 00:01:02,013 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/6c729f49e6ad4e8abeab467620b66144_SeqId_49_ into f8230c1f516a2a5268355abb24416354/family1 2023-10-28 00:01:02,013 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMinorCompactionWithMonthlyPolicy__r8sbl93v408q2v4c281vuab0p19utck0qjrnalf51kdq2ikv7ilcjbk2k9g89i57/family1/883a123cebb849e0bf8b64966598a09c into f8230c1f516a2a5268355abb24416354/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/6c729f49e6ad4e8abeab467620b66144_SeqId_49_) 2023-10-28 00:01:02,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMinorCompactionWithMonthlyPolicy__r8sbl93v408q2v4c281vuab0p19utck0qjrnalf51kdq2ikv7ilcjbk2k9g89i57/family1/883a123cebb849e0bf8b64966598a09c 2023-10-28 00:01:02,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-10-28 00:01:02,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 352581 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-10-28 00:01:02,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(1912): f8230c1f516a2a5268355abb24416354/family1 is initiating minor compaction (all files) 2023-10-28 00:01:02,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@277e068d; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-28 00:01:02,014 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of f8230c1f516a2a5268355abb24416354/family1 in testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:01:02,015 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/d9852f596cdf4260a6cc2e4cea7a5a8a, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/84e9e6efba454b689c20f22510e2f783, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/ebf2556d4dfb492591fbd6cb65b54c7c_SeqId_46_, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/6c729f49e6ad4e8abeab467620b66144_SeqId_49_] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/.tmp, totalSize=344.3 K 2023-10-28 00:01:02,015 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting d9852f596cdf4260a6cc2e4cea7a5a8a, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878821089 2023-10-28 00:01:02,016 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 84e9e6efba454b689c20f22510e2f783, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621221089 2023-10-28 00:01:02,016 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-10-28 00:01:02,016 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting ebf2556d4dfb492591fbd6cb65b54c7c_SeqId_46_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1448965221089 2023-10-28 00:01:02,017 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:01:02,017 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] compactions.Compactor(207): Compacting 6c729f49e6ad4e8abeab467620b66144_SeqId_49_, keycount=1001, bloomtype=ROW, size=166.2 K, encoding=NONE, compression=NONE, seqNum=50, earliestPutTs=1448014821089 2023-10-28 00:01:02,018 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511205eea7cb6af214277809843d8d7de1e15 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511205eea7cb6af214277809843d8d7de1e15 2023-10-28 00:01:02,019 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128e147d844a6cb4a998f384b845a63c4ab to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128e147d844a6cb4a998f384b845a63c4ab 2023-10-28 00:01:02,020 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511309cb1346e939f49a882abda6bda6914fc to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511309cb1346e939f49a882abda6bda6914fc 2023-10-28 00:01:02,020 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-10-28 00:01:02,020 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-10-28 00:01:02,021 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-10-28 00:01:02,021 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-10-28 00:01:02,021 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-10-28 00:01:02,021 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-10-28 00:01:02,021 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-28 00:01:02,023 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-10-28 00:01:02,024 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.9010 sec 2023-10-28 00:01:02,033 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] throttle.PressureAwareThroughputController(145): f8230c1f516a2a5268355abb24416354#family1#compaction#167 average throughput is 25.93 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 57.69 MB/second 2023-10-28 00:01:02,194 INFO [Listener at localhost/40295] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=259 (was 249) - Thread LEAK? -, OpenFileDescriptor=649 (was 632) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=50 (was 64), ProcessCount=168 (was 168), AvailableMemoryMB=5897 (was 5905) 2023-10-28 00:01:02,204 INFO [Listener at localhost/40295] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=259, OpenFileDescriptor=649, MaxFileDescriptor=60000, SystemLoadAverage=50, ProcessCount=168, AvailableMemoryMB=5897 2023-10-28 00:01:02,221 DEBUG [Listener at localhost/40295] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-10-28 00:01:02,263 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.HMaster$4(2112): Client=jenkins//188.40.62.62 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x97\x0E\xBE`\xA8\xAF\xBF1\x9Ee\xEB\xFFs6\xFD\xB2"\x10\x97h\x83E\xCA\x8C\xC8Q\xBDn\xEAi\xE9\xE2\xE7\xA6*\x10\xE3\x1A\x91\xE8\x0Bx\xF2\x90\xF6\xCC\x84\xF9\x0D\x09\xCD\xFA2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-10-28 00:01:02,264 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure2.ProcedureExecutor(1028): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-10-28 00:01:02,265 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-10-28 00:01:02,265 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(700): Client=jenkins//188.40.62.62 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-10-28 00:01:02,266 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-10-28 00:01:02,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-10-28 00:01:02,268 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c 2023-10-28 00:01:02,268 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428 2023-10-28 00:01:02,268 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6 2023-10-28 00:01:02,268 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c empty. 2023-10-28 00:01:02,268 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6 empty. 2023-10-28 00:01:02,268 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428 empty. 2023-10-28 00:01:02,268 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c 2023-10-28 00:01:02,268 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6 2023-10-28 00:01:02,268 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428 2023-10-28 00:01:02,269 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-10-28 00:01:02,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-10-28 00:01:02,569 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-10-28 00:01:02,677 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-10-28 00:01:02,679 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7675): creating {ENCODED => 1b31b523f7e2913dd7d63aebefeb7c9c, NAME => 'testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x97\x0E\xBE`\xA8\xAF\xBF1\x9Ee\xEB\xFFs6\xFD\xB2"\x10\x97h\x83E\xCA\x8C\xC8Q\xBDn\xEAi\xE9\xE2\xE7\xA6*\x10\xE3\x1A\x91\xE8\x0Bx\xF2\x90\xF6\xCC\x84\xF9\x0D\x09\xCD\xFA2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp 2023-10-28 00:01:02,680 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7675): creating {ENCODED => 136ee2e7ad6094dbdbe7b527030e07b6, NAME => 'testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x97\x0E\xBE`\xA8\xAF\xBF1\x9Ee\xEB\xFFs6\xFD\xB2"\x10\x97h\x83E\xCA\x8C\xC8Q\xBDn\xEAi\xE9\xE2\xE7\xA6*\x10\xE3\x1A\x91\xE8\x0Bx\xF2\x90\xF6\xCC\x84\xF9\x0D\x09\xCD\xFA2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp 2023-10-28 00:01:02,681 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7675): creating {ENCODED => fdbee376513d41f5e7553c6988d70428, NAME => 'testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x97\x0E\xBE`\xA8\xAF\xBF1\x9Ee\xEB\xFFs6\xFD\xB2"\x10\x97h\x83E\xCA\x8C\xC8Q\xBDn\xEAi\xE9\xE2\xE7\xA6*\x10\xE3\x1A\x91\xE8\x0Bx\xF2\x90\xF6\xCC\x84\xF9\x0D\x09\xCD\xFA2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/.tmp 2023-10-28 00:01:02,867 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/.tmp/family1/78f09cc10abe4452bdb6f15b706767c1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/78f09cc10abe4452bdb6f15b706767c1 2023-10-28 00:01:02,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-10-28 00:01:02,873 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in f8230c1f516a2a5268355abb24416354/family1 of f8230c1f516a2a5268355abb24416354 into 78f09cc10abe4452bdb6f15b706767c1(size=167.4 K), total size for store is 167.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-10-28 00:01:02,873 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for f8230c1f516a2a5268355abb24416354: 2023-10-28 00:01:02,873 INFO [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354., storeName=family1, fileCount=4, fileSize=344.3 K (166.0 K, 6.6 K, 5.5 K, 166.2 K), priority=2, time=1698451262014; duration=0sec 2023-10-28 00:01:02,873 DEBUG [RS:0;jenkins-hbase16:43469-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-28 00:01:02,929 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/d9852f596cdf4260a6cc2e4cea7a5a8a, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/84e9e6efba454b689c20f22510e2f783, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/ebf2556d4dfb492591fbd6cb65b54c7c_SeqId_46_, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/6c729f49e6ad4e8abeab467620b66144_SeqId_49_] to archive 2023-10-28 00:01:02,956 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:01:02,959 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/d9852f596cdf4260a6cc2e4cea7a5a8a to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/d9852f596cdf4260a6cc2e4cea7a5a8a 2023-10-28 00:01:02,961 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/84e9e6efba454b689c20f22510e2f783 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/84e9e6efba454b689c20f22510e2f783 2023-10-28 00:01:02,963 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/ebf2556d4dfb492591fbd6cb65b54c7c_SeqId_46_ to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/ebf2556d4dfb492591fbd6cb65b54c7c_SeqId_46_ 2023-10-28 00:01:02,965 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/6c729f49e6ad4e8abeab467620b66144_SeqId_49_ to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/family1/6c729f49e6ad4e8abeab467620b66144_SeqId_49_ 2023-10-28 00:01:03,098 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-28 00:01:03,098 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-28 00:01:03,098 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-28 00:01:03,098 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1604): Closing 136ee2e7ad6094dbdbe7b527030e07b6, disabling compactions & flushes 2023-10-28 00:01:03,099 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. 2023-10-28 00:01:03,099 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. 2023-10-28 00:01:03,099 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. after waiting 0 ms 2023-10-28 00:01:03,099 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. 2023-10-28 00:01:03,099 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. 2023-10-28 00:01:03,099 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1558): Region close journal for 136ee2e7ad6094dbdbe7b527030e07b6: 2023-10-28 00:01:03,098 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1604): Closing fdbee376513d41f5e7553c6988d70428, disabling compactions & flushes 2023-10-28 00:01:03,100 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. 2023-10-28 00:01:03,100 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. 2023-10-28 00:01:03,100 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. after waiting 0 ms 2023-10-28 00:01:03,100 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. 2023-10-28 00:01:03,099 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1604): Closing 1b31b523f7e2913dd7d63aebefeb7c9c, disabling compactions & flushes 2023-10-28 00:01:03,100 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. 2023-10-28 00:01:03,100 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. 2023-10-28 00:01:03,100 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1558): Region close journal for fdbee376513d41f5e7553c6988d70428: 2023-10-28 00:01:03,101 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. 2023-10-28 00:01:03,101 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. after waiting 0 ms 2023-10-28 00:01:03,101 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. 2023-10-28 00:01:03,101 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. 2023-10-28 00:01:03,101 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1558): Region close journal for 1b31b523f7e2913dd7d63aebefeb7c9c: 2023-10-28 00:01:03,104 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-10-28 00:01:03,105 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1698451263105"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451263105"}]},"ts":"1698451263105"} 2023-10-28 00:01:03,105 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1698451263105"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451263105"}]},"ts":"1698451263105"} 2023-10-28 00:01:03,105 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1698451263105"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698451263105"}]},"ts":"1698451263105"} 2023-10-28 00:01:03,107 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-10-28 00:01:03,107 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-10-28 00:01:03,107 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451263107"}]},"ts":"1698451263107"} 2023-10-28 00:01:03,108 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-10-28 00:01:03,130 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=1b31b523f7e2913dd7d63aebefeb7c9c, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=fdbee376513d41f5e7553c6988d70428, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=136ee2e7ad6094dbdbe7b527030e07b6, ASSIGN}] 2023-10-28 00:01:03,131 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=fdbee376513d41f5e7553c6988d70428, ASSIGN 2023-10-28 00:01:03,131 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=136ee2e7ad6094dbdbe7b527030e07b6, ASSIGN 2023-10-28 00:01:03,132 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=1b31b523f7e2913dd7d63aebefeb7c9c, ASSIGN 2023-10-28 00:01:03,132 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=136ee2e7ad6094dbdbe7b527030e07b6, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,43469,1698451096272; forceNewPlan=false, retain=false 2023-10-28 00:01:03,132 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=fdbee376513d41f5e7553c6988d70428, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,43469,1698451096272; forceNewPlan=false, retain=false 2023-10-28 00:01:03,132 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=1b31b523f7e2913dd7d63aebefeb7c9c, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,43469,1698451096272; forceNewPlan=false, retain=false 2023-10-28 00:01:03,286 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=136ee2e7ad6094dbdbe7b527030e07b6, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:03,286 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=1b31b523f7e2913dd7d63aebefeb7c9c, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:03,286 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1698451263286"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451263286"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451263286"}]},"ts":"1698451263286"} 2023-10-28 00:01:03,286 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1698451263286"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451263286"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451263286"}]},"ts":"1698451263286"} 2023-10-28 00:01:03,286 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=fdbee376513d41f5e7553c6988d70428, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:03,287 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1698451263286"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1698451263286"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698451263286"}]},"ts":"1698451263286"} 2023-10-28 00:01:03,289 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE; OpenRegionProcedure 136ee2e7ad6094dbdbe7b527030e07b6, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-28 00:01:03,290 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=121, ppid=117, state=RUNNABLE; OpenRegionProcedure 1b31b523f7e2913dd7d63aebefeb7c9c, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-28 00:01:03,292 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=122, ppid=118, state=RUNNABLE; OpenRegionProcedure fdbee376513d41f5e7553c6988d70428, server=jenkins-hbase16.apache.org,43469,1698451096272}] 2023-10-28 00:01:03,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-10-28 00:01:03,443 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:03,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. 2023-10-28 00:01:03,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 136ee2e7ad6094dbdbe7b527030e07b6, NAME => 'testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6.', STARTKEY => '2', ENDKEY => ''} 2023-10-28 00:01:03,450 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 136ee2e7ad6094dbdbe7b527030e07b6 2023-10-28 00:01:03,450 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-28 00:01:03,450 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for 136ee2e7ad6094dbdbe7b527030e07b6 2023-10-28 00:01:03,450 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for 136ee2e7ad6094dbdbe7b527030e07b6 2023-10-28 00:01:03,473 INFO [StoreOpener-136ee2e7ad6094dbdbe7b527030e07b6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 136ee2e7ad6094dbdbe7b527030e07b6 2023-10-28 00:01:03,474 DEBUG [StoreOpener-136ee2e7ad6094dbdbe7b527030e07b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1 2023-10-28 00:01:03,474 DEBUG [StoreOpener-136ee2e7ad6094dbdbe7b527030e07b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1 2023-10-28 00:01:03,474 INFO [StoreOpener-136ee2e7ad6094dbdbe7b527030e07b6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 136ee2e7ad6094dbdbe7b527030e07b6 columnFamilyName family1 2023-10-28 00:01:03,476 INFO [StoreOpener-136ee2e7ad6094dbdbe7b527030e07b6-1] regionserver.HStore(310): Store=136ee2e7ad6094dbdbe7b527030e07b6/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-28 00:01:03,476 INFO [StoreOpener-136ee2e7ad6094dbdbe7b527030e07b6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 136ee2e7ad6094dbdbe7b527030e07b6 2023-10-28 00:01:03,477 DEBUG [StoreOpener-136ee2e7ad6094dbdbe7b527030e07b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2 2023-10-28 00:01:03,477 DEBUG [StoreOpener-136ee2e7ad6094dbdbe7b527030e07b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2 2023-10-28 00:01:03,477 INFO [StoreOpener-136ee2e7ad6094dbdbe7b527030e07b6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 136ee2e7ad6094dbdbe7b527030e07b6 columnFamilyName family2 2023-10-28 00:01:03,478 INFO [StoreOpener-136ee2e7ad6094dbdbe7b527030e07b6-1] regionserver.HStore(310): Store=136ee2e7ad6094dbdbe7b527030e07b6/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-28 00:01:03,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6 2023-10-28 00:01:03,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6 2023-10-28 00:01:03,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-10-28 00:01:03,481 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for 136ee2e7ad6094dbdbe7b527030e07b6 2023-10-28 00:01:03,482 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-10-28 00:01:03,482 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened 136ee2e7ad6094dbdbe7b527030e07b6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64669740, jitterRate=-0.03634577989578247}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-10-28 00:01:03,482 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for 136ee2e7ad6094dbdbe7b527030e07b6: 2023-10-28 00:01:03,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6., pid=120, masterSystemTime=1698451263443 2023-10-28 00:01:03,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. 2023-10-28 00:01:03,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. 2023-10-28 00:01:03,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. 2023-10-28 00:01:03,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => fdbee376513d41f5e7553c6988d70428, NAME => 'testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428.', STARTKEY => '1', ENDKEY => '2'} 2023-10-28 00:01:03,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin fdbee376513d41f5e7553c6988d70428 2023-10-28 00:01:03,484 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=136ee2e7ad6094dbdbe7b527030e07b6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:03,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-28 00:01:03,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for fdbee376513d41f5e7553c6988d70428 2023-10-28 00:01:03,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for fdbee376513d41f5e7553c6988d70428 2023-10-28 00:01:03,484 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1698451263484"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451263484"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451263484"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451263484"}]},"ts":"1698451263484"} 2023-10-28 00:01:03,486 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=120, resume processing ppid=119 2023-10-28 00:01:03,486 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=120, ppid=119, state=SUCCESS; OpenRegionProcedure 136ee2e7ad6094dbdbe7b527030e07b6, server=jenkins-hbase16.apache.org,43469,1698451096272 in 196 msec 2023-10-28 00:01:03,487 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=136ee2e7ad6094dbdbe7b527030e07b6, ASSIGN in 357 msec 2023-10-28 00:01:03,504 INFO [StoreOpener-fdbee376513d41f5e7553c6988d70428-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region fdbee376513d41f5e7553c6988d70428 2023-10-28 00:01:03,505 DEBUG [StoreOpener-fdbee376513d41f5e7553c6988d70428-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1 2023-10-28 00:01:03,506 DEBUG [StoreOpener-fdbee376513d41f5e7553c6988d70428-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1 2023-10-28 00:01:03,506 INFO [StoreOpener-fdbee376513d41f5e7553c6988d70428-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fdbee376513d41f5e7553c6988d70428 columnFamilyName family1 2023-10-28 00:01:03,507 INFO [StoreOpener-fdbee376513d41f5e7553c6988d70428-1] regionserver.HStore(310): Store=fdbee376513d41f5e7553c6988d70428/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-28 00:01:03,507 INFO [StoreOpener-fdbee376513d41f5e7553c6988d70428-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region fdbee376513d41f5e7553c6988d70428 2023-10-28 00:01:03,508 DEBUG [StoreOpener-fdbee376513d41f5e7553c6988d70428-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2 2023-10-28 00:01:03,508 DEBUG [StoreOpener-fdbee376513d41f5e7553c6988d70428-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2 2023-10-28 00:01:03,508 INFO [StoreOpener-fdbee376513d41f5e7553c6988d70428-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fdbee376513d41f5e7553c6988d70428 columnFamilyName family2 2023-10-28 00:01:03,509 INFO [StoreOpener-fdbee376513d41f5e7553c6988d70428-1] regionserver.HStore(310): Store=fdbee376513d41f5e7553c6988d70428/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-28 00:01:03,510 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428 2023-10-28 00:01:03,510 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428 2023-10-28 00:01:03,511 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-10-28 00:01:03,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for fdbee376513d41f5e7553c6988d70428 2023-10-28 00:01:03,513 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-10-28 00:01:03,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened fdbee376513d41f5e7553c6988d70428; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67092224, jitterRate=-2.47955322265625E-4}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-10-28 00:01:03,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for fdbee376513d41f5e7553c6988d70428: 2023-10-28 00:01:03,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428., pid=122, masterSystemTime=1698451263443 2023-10-28 00:01:03,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. 2023-10-28 00:01:03,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. 2023-10-28 00:01:03,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. 2023-10-28 00:01:03,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1b31b523f7e2913dd7d63aebefeb7c9c, NAME => 'testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c.', STARTKEY => '', ENDKEY => '1'} 2023-10-28 00:01:03,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 1b31b523f7e2913dd7d63aebefeb7c9c 2023-10-28 00:01:03,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-10-28 00:01:03,516 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=fdbee376513d41f5e7553c6988d70428, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:03,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7894): checking encryption for 1b31b523f7e2913dd7d63aebefeb7c9c 2023-10-28 00:01:03,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7897): checking classloading for 1b31b523f7e2913dd7d63aebefeb7c9c 2023-10-28 00:01:03,516 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1698451263516"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451263516"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451263516"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451263516"}]},"ts":"1698451263516"} 2023-10-28 00:01:03,518 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=122, resume processing ppid=118 2023-10-28 00:01:03,518 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=122, ppid=118, state=SUCCESS; OpenRegionProcedure fdbee376513d41f5e7553c6988d70428, server=jenkins-hbase16.apache.org,43469,1698451096272 in 225 msec 2023-10-28 00:01:03,519 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=fdbee376513d41f5e7553c6988d70428, ASSIGN in 389 msec 2023-10-28 00:01:03,536 INFO [StoreOpener-1b31b523f7e2913dd7d63aebefeb7c9c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1b31b523f7e2913dd7d63aebefeb7c9c 2023-10-28 00:01:03,537 DEBUG [StoreOpener-1b31b523f7e2913dd7d63aebefeb7c9c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1 2023-10-28 00:01:03,537 DEBUG [StoreOpener-1b31b523f7e2913dd7d63aebefeb7c9c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1 2023-10-28 00:01:03,537 INFO [StoreOpener-1b31b523f7e2913dd7d63aebefeb7c9c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1b31b523f7e2913dd7d63aebefeb7c9c columnFamilyName family1 2023-10-28 00:01:03,538 INFO [StoreOpener-1b31b523f7e2913dd7d63aebefeb7c9c-1] regionserver.HStore(310): Store=1b31b523f7e2913dd7d63aebefeb7c9c/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-28 00:01:03,538 INFO [StoreOpener-1b31b523f7e2913dd7d63aebefeb7c9c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 1b31b523f7e2913dd7d63aebefeb7c9c 2023-10-28 00:01:03,539 DEBUG [StoreOpener-1b31b523f7e2913dd7d63aebefeb7c9c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2 2023-10-28 00:01:03,539 DEBUG [StoreOpener-1b31b523f7e2913dd7d63aebefeb7c9c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2 2023-10-28 00:01:03,539 INFO [StoreOpener-1b31b523f7e2913dd7d63aebefeb7c9c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1b31b523f7e2913dd7d63aebefeb7c9c columnFamilyName family2 2023-10-28 00:01:03,540 INFO [StoreOpener-1b31b523f7e2913dd7d63aebefeb7c9c-1] regionserver.HStore(310): Store=1b31b523f7e2913dd7d63aebefeb7c9c/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-28 00:01:03,541 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c 2023-10-28 00:01:03,541 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c 2023-10-28 00:01:03,542 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-10-28 00:01:03,543 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1055): writing seq id for 1b31b523f7e2913dd7d63aebefeb7c9c 2023-10-28 00:01:03,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-10-28 00:01:03,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1072): Opened 1b31b523f7e2913dd7d63aebefeb7c9c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71405561, jitterRate=0.06402577459812164}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-10-28 00:01:03,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(965): Region open journal for 1b31b523f7e2913dd7d63aebefeb7c9c: 2023-10-28 00:01:03,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c., pid=121, masterSystemTime=1698451263443 2023-10-28 00:01:03,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. 2023-10-28 00:01:03,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. 2023-10-28 00:01:03,547 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=1b31b523f7e2913dd7d63aebefeb7c9c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:03,547 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1698451263547"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1698451263547"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698451263547"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698451263547"}]},"ts":"1698451263547"} 2023-10-28 00:01:03,549 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=121, resume processing ppid=117 2023-10-28 00:01:03,549 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=121, ppid=117, state=SUCCESS; OpenRegionProcedure 1b31b523f7e2913dd7d63aebefeb7c9c, server=jenkins-hbase16.apache.org,43469,1698451096272 in 258 msec 2023-10-28 00:01:03,550 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=117, resume processing ppid=116 2023-10-28 00:01:03,550 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=1b31b523f7e2913dd7d63aebefeb7c9c, ASSIGN in 420 msec 2023-10-28 00:01:03,550 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-10-28 00:01:03,550 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698451263550"}]},"ts":"1698451263550"} 2023-10-28 00:01:03,551 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-10-28 00:01:03,604 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-10-28 00:01:03,607 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 1.3410 sec 2023-10-28 00:01:04,374 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-10-28 00:01:04,375 INFO [Listener at localhost/40295] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-10-28 00:01:04,386 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:01:04,387 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:01:04,387 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:01:04,390 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-28 00:01:04,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-10-28 00:01:04,391 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-10-28 00:01:04,391 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:01:04,391 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-10-28 00:01:04,391 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-10-28 00:01:04,392 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:04,392 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-10-28 00:01:04,427 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:01:04,428 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:04,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:01:04,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:01:04,428 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:04,428 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-28 00:01:04,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-10-28 00:01:04,429 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:04,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-28 00:01:04,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-10-28 00:01:04,430 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-10-28 00:01:04,430 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-10-28 00:01:04,430 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-10-28 00:01:04,430 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:01:04,430 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-10-28 00:01:04,431 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. 2023-10-28 00:01:04,431 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. started... 2023-10-28 00:01:04,431 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. 2023-10-28 00:01:04,431 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. 2023-10-28 00:01:04,432 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. started... 2023-10-28 00:01:04,431 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-10-28 00:01:04,433 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-28 00:01:04,433 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 1b31b523f7e2913dd7d63aebefeb7c9c 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-10-28 00:01:04,432 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 136ee2e7ad6094dbdbe7b527030e07b6 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-10-28 00:01:04,431 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. started... 2023-10-28 00:01:04,434 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing fdbee376513d41f5e7553c6988d70428 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-10-28 00:01:04,476 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202310283083d25633e44fe5934ae7a6eb2c1841 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202310283083d25633e44fe5934ae7a6eb2c1841 2023-10-28 00:01:04,888 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231028dfa1b2d5025843be858ca1b2716f2bd4 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231028dfa1b2d5025843be858ca1b2716f2bd4 2023-10-28 00:01:04,888 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023102892feee79f01c4fbd9d9510c42ca4c34b to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023102892feee79f01c4fbd9d9510c42ca4c34b 2023-10-28 00:01:04,889 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/.tmp/family1/4b0eb47973cf4ec198a45ca8917cf70b 2023-10-28 00:01:05,302 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/.tmp/family1/c277995f9d4449afba51b623b755bd2c 2023-10-28 00:01:05,313 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/.tmp/family1/86adf2c58f924b709e65532688c3462d 2023-10-28 00:01:05,324 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231028551814db106c4b6294d63b596df2380e to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20231028551814db106c4b6294d63b596df2380e 2023-10-28 00:01:05,328 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/.tmp/family2/30fbf6fe0f364762b0c7d67d64a97307 2023-10-28 00:01:05,332 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/.tmp/family1/4b0eb47973cf4ec198a45ca8917cf70b as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/4b0eb47973cf4ec198a45ca8917cf70b 2023-10-28 00:01:05,355 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231028743ca7e5502a4f8c965a2ad3e517a68a to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20231028743ca7e5502a4f8c965a2ad3e517a68a 2023-10-28 00:01:05,355 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/4b0eb47973cf4ec198a45ca8917cf70b, entries=4, sequenceid=5, filesize=5.5 K 2023-10-28 00:01:05,356 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/.tmp/family2/30fbf6fe0f364762b0c7d67d64a97307 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/30fbf6fe0f364762b0c7d67d64a97307 2023-10-28 00:01:05,359 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/30fbf6fe0f364762b0c7d67d64a97307, entries=2, sequenceid=5, filesize=5.1 K 2023-10-28 00:01:05,360 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for fdbee376513d41f5e7553c6988d70428 in 926ms, sequenceid=5, compaction requested=false 2023-10-28 00:01:05,360 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-10-28 00:01:05,360 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for fdbee376513d41f5e7553c6988d70428: 2023-10-28 00:01:05,360 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. 2023-10-28 00:01:05,360 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-10-28 00:01:05,755 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023102887b64e3d7c784f69a38cb34b37cdb0f1 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c2023102887b64e3d7c784f69a38cb34b37cdb0f1 2023-10-28 00:01:05,759 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/.tmp/family2/c64f217fda7948d3beefb7b507c68d95 2023-10-28 00:01:05,764 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/.tmp/family1/86adf2c58f924b709e65532688c3462d as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/86adf2c58f924b709e65532688c3462d 2023-10-28 00:01:05,790 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/86adf2c58f924b709e65532688c3462d, entries=4, sequenceid=5, filesize=5.5 K 2023-10-28 00:01:05,791 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/.tmp/family2/c64f217fda7948d3beefb7b507c68d95 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/c64f217fda7948d3beefb7b507c68d95 2023-10-28 00:01:05,794 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/c64f217fda7948d3beefb7b507c68d95, entries=2, sequenceid=5, filesize=5.1 K 2023-10-28 00:01:05,795 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 1b31b523f7e2913dd7d63aebefeb7c9c in 1362ms, sequenceid=5, compaction requested=false 2023-10-28 00:01:05,795 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 1b31b523f7e2913dd7d63aebefeb7c9c: 2023-10-28 00:01:05,796 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. 2023-10-28 00:01:05,796 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-10-28 00:01:06,162 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/.tmp/family2/9e32ab004ceb44eca3c759b2b6241ec9 2023-10-28 00:01:06,166 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/.tmp/family1/c277995f9d4449afba51b623b755bd2c as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/c277995f9d4449afba51b623b755bd2c 2023-10-28 00:01:06,188 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/c277995f9d4449afba51b623b755bd2c, entries=4, sequenceid=5, filesize=5.5 K 2023-10-28 00:01:06,189 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/.tmp/family2/9e32ab004ceb44eca3c759b2b6241ec9 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/9e32ab004ceb44eca3c759b2b6241ec9 2023-10-28 00:01:06,193 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/9e32ab004ceb44eca3c759b2b6241ec9, entries=2, sequenceid=5, filesize=5.1 K 2023-10-28 00:01:06,193 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 136ee2e7ad6094dbdbe7b527030e07b6 in 1761ms, sequenceid=5, compaction requested=false 2023-10-28 00:01:06,193 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 136ee2e7ad6094dbdbe7b527030e07b6: 2023-10-28 00:01:06,193 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. 2023-10-28 00:01:06,193 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-10-28 00:01:06,194 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-10-28 00:01:06,194 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:06,194 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-10-28 00:01:06,194 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-10-28 00:01:06,236 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:06,236 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:06,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:06,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:01:06,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:01:06,237 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:06,237 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-28 00:01:06,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:01:06,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:01:06,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:06,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:06,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:01:06,240 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-10-28 00:01:06,240 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@244e23c0[Count = 0] remaining members to acquire global barrier 2023-10-28 00:01:06,240 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-10-28 00:01:06,240 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:06,246 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:06,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:06,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:06,246 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-10-28 00:01:06,246 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-10-28 00:01:06,246 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-28 00:01:06,247 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:06,247 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-28 00:01:06,254 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-10-28 00:01:06,254 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:06,254 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:01:06,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:06,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:01:06,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:01:06,255 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-10-28 00:01:06,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:01:06,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:01:06,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:06,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:06,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:01:06,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:06,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:06,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-28 00:01:06,262 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-10-28 00:01:06,262 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-10-28 00:01:06,262 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-28 00:01:06,262 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-28 00:01:06,262 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-10-28 00:01:06,262 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-28 00:01:06,271 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:06,271 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:06,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:06,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:01:06,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:01:06,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:06,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:06,271 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:01:06,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:01:06,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:01:06,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:01:06,273 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:06,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:06,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:06,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:01:06,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:06,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:06,275 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:06,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:01:06,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:06,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:06,288 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:06,288 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:01:06,288 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-10-28 00:01:06,288 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-10-28 00:01:06,288 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-10-28 00:01:06,288 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-28 00:01:06,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:01:06,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:01:06,288 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:01:06,288 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:06,289 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-10-28 00:01:06,288 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:01:06,289 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-28 00:01:06,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:01:06,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:01:06,289 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:06,290 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:06,290 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:06,390 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-28 00:01:06,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-28 00:01:06,394 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:01:06,394 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:01:06,395 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:01:06,401 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-28 00:01:06,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-10-28 00:01:06,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-10-28 00:01:06,404 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-10-28 00:01:06,404 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:01:06,405 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-10-28 00:01:06,405 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-10-28 00:01:06,405 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:06,406 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-10-28 00:01:06,413 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:01:06,413 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:06,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:01:06,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:01:06,413 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:06,413 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-28 00:01:06,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-10-28 00:01:06,414 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:06,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-28 00:01:06,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-10-28 00:01:06,414 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-10-28 00:01:06,415 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-10-28 00:01:06,415 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-10-28 00:01:06,415 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-10-28 00:01:06,415 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:01:06,415 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-10-28 00:01:06,415 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. 2023-10-28 00:01:06,415 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. started... 2023-10-28 00:01:06,415 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. 2023-10-28 00:01:06,415 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. started... 2023-10-28 00:01:06,415 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. 2023-10-28 00:01:06,415 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing fdbee376513d41f5e7553c6988d70428 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-10-28 00:01:06,415 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-10-28 00:01:06,416 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-28 00:01:06,415 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. started... 2023-10-28 00:01:06,415 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 136ee2e7ad6094dbdbe7b527030e07b6 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-10-28 00:01:06,416 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 1b31b523f7e2913dd7d63aebefeb7c9c 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-10-28 00:01:06,463 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202310286d07ffc33ed04d47b18795f47f483eb4 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202310286d07ffc33ed04d47b18795f47f483eb4 2023-10-28 00:01:06,472 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231028dd2caa04067743be96420810eb0ab746 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231028dd2caa04067743be96420810eb0ab746 2023-10-28 00:01:06,481 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231028d20a43169ea2437690666276cab2c746 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231028d20a43169ea2437690666276cab2c746 2023-10-28 00:01:06,489 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/.tmp/family1/34a67289c63e4e8593c3d6fc7848be8c 2023-10-28 00:01:06,881 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/.tmp/family1/f630f37ee0ab4f0e862cd74c9318a717 2023-10-28 00:01:06,895 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/.tmp/family1/720a86219f5d474185e2c4182d8e747f 2023-10-28 00:01:06,918 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202310283aa0c887a93140e29d916e5b3dac9b4c to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202310283aa0c887a93140e29d916e5b3dac9b4c 2023-10-28 00:01:07,322 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231028b0acb64adc0a441ab7f6340da594e377 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20231028b0acb64adc0a441ab7f6340da594e377 2023-10-28 00:01:07,322 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/.tmp/family2/a1c30a3fbaa545cc9c0a048666af383d 2023-10-28 00:01:07,329 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202310282dd32fcf0f484630a0d1f203e131afd5 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202310282dd32fcf0f484630a0d1f203e131afd5 2023-10-28 00:01:07,329 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/.tmp/family2/cac7b6e36f824fabb794921565bf4d31 2023-10-28 00:01:07,330 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/.tmp/family1/34a67289c63e4e8593c3d6fc7848be8c as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/34a67289c63e4e8593c3d6fc7848be8c 2023-10-28 00:01:07,334 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/.tmp/family1/f630f37ee0ab4f0e862cd74c9318a717 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/f630f37ee0ab4f0e862cd74c9318a717 2023-10-28 00:01:07,334 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/.tmp/family2/949be6eeb7e94f4aa18d82b789a298b8 2023-10-28 00:01:07,338 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/.tmp/family1/720a86219f5d474185e2c4182d8e747f as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/720a86219f5d474185e2c4182d8e747f 2023-10-28 00:01:07,356 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/34a67289c63e4e8593c3d6fc7848be8c, entries=4, sequenceid=9, filesize=5.5 K 2023-10-28 00:01:07,357 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/f630f37ee0ab4f0e862cd74c9318a717, entries=4, sequenceid=9, filesize=5.5 K 2023-10-28 00:01:07,357 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/720a86219f5d474185e2c4182d8e747f, entries=4, sequenceid=9, filesize=5.5 K 2023-10-28 00:01:07,359 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/.tmp/family2/949be6eeb7e94f4aa18d82b789a298b8 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/949be6eeb7e94f4aa18d82b789a298b8 2023-10-28 00:01:07,359 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/.tmp/family2/a1c30a3fbaa545cc9c0a048666af383d as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/a1c30a3fbaa545cc9c0a048666af383d 2023-10-28 00:01:07,359 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/.tmp/family2/cac7b6e36f824fabb794921565bf4d31 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/cac7b6e36f824fabb794921565bf4d31 2023-10-28 00:01:07,363 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/949be6eeb7e94f4aa18d82b789a298b8, entries=2, sequenceid=9, filesize=5.1 K 2023-10-28 00:01:07,363 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/a1c30a3fbaa545cc9c0a048666af383d, entries=2, sequenceid=9, filesize=5.1 K 2023-10-28 00:01:07,363 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/cac7b6e36f824fabb794921565bf4d31, entries=2, sequenceid=9, filesize=5.1 K 2023-10-28 00:01:07,363 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for fdbee376513d41f5e7553c6988d70428 in 948ms, sequenceid=9, compaction requested=false 2023-10-28 00:01:07,363 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 136ee2e7ad6094dbdbe7b527030e07b6 in 948ms, sequenceid=9, compaction requested=false 2023-10-28 00:01:07,363 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for fdbee376513d41f5e7553c6988d70428: 2023-10-28 00:01:07,363 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 136ee2e7ad6094dbdbe7b527030e07b6: 2023-10-28 00:01:07,363 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. 2023-10-28 00:01:07,364 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 1b31b523f7e2913dd7d63aebefeb7c9c in 948ms, sequenceid=9, compaction requested=false 2023-10-28 00:01:07,364 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-10-28 00:01:07,364 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. 2023-10-28 00:01:07,364 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 1b31b523f7e2913dd7d63aebefeb7c9c: 2023-10-28 00:01:07,364 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-10-28 00:01:07,364 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. 2023-10-28 00:01:07,364 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-10-28 00:01:07,364 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-10-28 00:01:07,364 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:07,364 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-10-28 00:01:07,364 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-10-28 00:01:07,396 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:07,396 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:07,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:07,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:01:07,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:01:07,397 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:07,397 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-28 00:01:07,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:01:07,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:01:07,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:07,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:07,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:01:07,398 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-10-28 00:01:07,398 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7d2b18f[Count = 0] remaining members to acquire global barrier 2023-10-28 00:01:07,398 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-10-28 00:01:07,398 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:07,404 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:07,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:07,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:07,404 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-10-28 00:01:07,404 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-10-28 00:01:07,404 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-28 00:01:07,404 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:07,405 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-28 00:01:07,412 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-10-28 00:01:07,412 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:01:07,413 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-10-28 00:01:07,412 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:07,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:07,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:01:07,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:01:07,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:01:07,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:01:07,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:07,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:07,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:01:07,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:07,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:07,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-28 00:01:07,415 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-10-28 00:01:07,415 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-10-28 00:01:07,415 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-28 00:01:07,415 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-28 00:01:07,415 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-10-28 00:01:07,416 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-28 00:01:07,421 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:07,421 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:07,421 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:01:07,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:07,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:01:07,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:01:07,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:07,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:07,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:01:07,422 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:07,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:01:07,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:01:07,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:07,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:07,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:01:07,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:07,423 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:07,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:07,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:01:07,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:07,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:07,429 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:07,429 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:01:07,429 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-10-28 00:01:07,429 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-10-28 00:01:07,429 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-10-28 00:01:07,429 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-28 00:01:07,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:01:07,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:01:07,429 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:01:07,429 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:01:07,430 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-10-28 00:01:07,430 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-28 00:01:07,429 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-10-28 00:01:07,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:01:07,430 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:07,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:01:07,430 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:07,430 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:07,430 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:07,530 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-28 00:01:07,532 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-28 00:01:07,534 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:01:07,534 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:01:07,535 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:01:07,542 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-28 00:01:07,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-10-28 00:01:07,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-10-28 00:01:07,545 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-10-28 00:01:07,545 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:01:07,545 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-10-28 00:01:07,545 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-10-28 00:01:07,546 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:07,546 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-10-28 00:01:07,554 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:01:07,554 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:07,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:01:07,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:01:07,555 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:07,555 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-28 00:01:07,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-10-28 00:01:07,556 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:07,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-28 00:01:07,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-10-28 00:01:07,556 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-10-28 00:01:07,557 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-10-28 00:01:07,557 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-10-28 00:01:07,557 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-10-28 00:01:07,557 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:01:07,557 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-10-28 00:01:07,557 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. 2023-10-28 00:01:07,557 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. started... 2023-10-28 00:01:07,557 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. 2023-10-28 00:01:07,557 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 136ee2e7ad6094dbdbe7b527030e07b6 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-10-28 00:01:07,557 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-10-28 00:01:07,557 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-28 00:01:07,557 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. 2023-10-28 00:01:07,557 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. started... 2023-10-28 00:01:07,558 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. started... 2023-10-28 00:01:07,558 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing fdbee376513d41f5e7553c6988d70428 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-10-28 00:01:07,558 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 1b31b523f7e2913dd7d63aebefeb7c9c 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-10-28 00:01:08,009 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023102810465475b92342dba5e0cabc6df67358 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023102810465475b92342dba5e0cabc6df67358 2023-10-28 00:01:08,019 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231028b935b7280f1f40b3943b19b24f37d105 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231028b935b7280f1f40b3943b19b24f37d105 2023-10-28 00:01:08,019 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023102843811a84fa3b4eaba7538fb89727895a to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023102843811a84fa3b4eaba7538fb89727895a 2023-10-28 00:01:08,422 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/.tmp/family1/36b6757862eb47028733e102a535f32c 2023-10-28 00:01:08,433 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/.tmp/family1/7b8246041c6d4d4781b4a79f95bf7b5d 2023-10-28 00:01:08,433 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/.tmp/family1/ba57df9023b74a44ab9211ff0e235ad3 2023-10-28 00:01:08,487 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202310289517492b352646439a5ebeac9ae18f0f to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202310289517492b352646439a5ebeac9ae18f0f 2023-10-28 00:01:08,488 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231028423dcfae1407413e91a51a0124d61ed1 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20231028423dcfae1407413e91a51a0124d61ed1 2023-10-28 00:01:08,490 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231028defc58c52e1e4dc8abaf3b45901c05b6 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20231028defc58c52e1e4dc8abaf3b45901c05b6 2023-10-28 00:01:08,495 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/.tmp/family2/a45f67a713b544418da5eb04140fc30e 2023-10-28 00:01:08,499 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/.tmp/family1/7b8246041c6d4d4781b4a79f95bf7b5d as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/7b8246041c6d4d4781b4a79f95bf7b5d 2023-10-28 00:01:08,500 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/.tmp/family2/ea6a77049ea6401b832c11c52220df30 2023-10-28 00:01:08,504 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/.tmp/family1/36b6757862eb47028733e102a535f32c as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/36b6757862eb47028733e102a535f32c 2023-10-28 00:01:08,521 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/7b8246041c6d4d4781b4a79f95bf7b5d, entries=4, sequenceid=13, filesize=5.5 K 2023-10-28 00:01:08,522 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/.tmp/family2/a45f67a713b544418da5eb04140fc30e as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/a45f67a713b544418da5eb04140fc30e 2023-10-28 00:01:08,524 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/36b6757862eb47028733e102a535f32c, entries=4, sequenceid=13, filesize=5.5 K 2023-10-28 00:01:08,525 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/.tmp/family2/ea6a77049ea6401b832c11c52220df30 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/ea6a77049ea6401b832c11c52220df30 2023-10-28 00:01:08,525 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/a45f67a713b544418da5eb04140fc30e, entries=2, sequenceid=13, filesize=5.1 K 2023-10-28 00:01:08,526 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 1b31b523f7e2913dd7d63aebefeb7c9c in 968ms, sequenceid=13, compaction requested=true 2023-10-28 00:01:08,526 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 1b31b523f7e2913dd7d63aebefeb7c9c: 2023-10-28 00:01:08,526 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. 2023-10-28 00:01:08,526 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-10-28 00:01:08,528 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/ea6a77049ea6401b832c11c52220df30, entries=2, sequenceid=13, filesize=5.1 K 2023-10-28 00:01:08,529 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 136ee2e7ad6094dbdbe7b527030e07b6 in 972ms, sequenceid=13, compaction requested=true 2023-10-28 00:01:08,529 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 136ee2e7ad6094dbdbe7b527030e07b6: 2023-10-28 00:01:08,529 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. 2023-10-28 00:01:08,529 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-10-28 00:01:08,909 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/.tmp/family2/2c2508a444b146abaec3416ac7c8b646 2023-10-28 00:01:08,933 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/.tmp/family1/ba57df9023b74a44ab9211ff0e235ad3 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/ba57df9023b74a44ab9211ff0e235ad3 2023-10-28 00:01:08,962 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/ba57df9023b74a44ab9211ff0e235ad3, entries=4, sequenceid=13, filesize=5.5 K 2023-10-28 00:01:08,962 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/.tmp/family2/2c2508a444b146abaec3416ac7c8b646 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/2c2508a444b146abaec3416ac7c8b646 2023-10-28 00:01:08,966 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/2c2508a444b146abaec3416ac7c8b646, entries=2, sequenceid=13, filesize=5.1 K 2023-10-28 00:01:08,966 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for fdbee376513d41f5e7553c6988d70428 in 1408ms, sequenceid=13, compaction requested=true 2023-10-28 00:01:08,967 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for fdbee376513d41f5e7553c6988d70428: 2023-10-28 00:01:08,967 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. 2023-10-28 00:01:08,967 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-10-28 00:01:08,967 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-10-28 00:01:08,967 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:08,967 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-10-28 00:01:08,967 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-10-28 00:01:08,979 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:08,979 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:08,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:08,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:01:08,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:01:08,980 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:08,980 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-28 00:01:08,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:01:08,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:01:08,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:08,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:08,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:01:08,982 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-10-28 00:01:08,982 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@672376c[Count = 0] remaining members to acquire global barrier 2023-10-28 00:01:08,982 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-10-28 00:01:08,982 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:08,987 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:08,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:08,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:08,988 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-10-28 00:01:08,988 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-10-28 00:01:08,988 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-28 00:01:08,988 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:08,988 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-28 00:01:08,996 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-10-28 00:01:08,996 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:08,996 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:01:08,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:08,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:01:08,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:01:08,996 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-10-28 00:01:08,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:01:08,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:01:08,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:08,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:08,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:01:09,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:09,001 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-10-28 00:01:09,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:09,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-28 00:01:09,002 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-10-28 00:01:09,002 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-10-28 00:01:09,002 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-28 00:01:09,002 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-28 00:01:09,002 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-10-28 00:01:09,002 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-28 00:01:09,012 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:09,012 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:01:09,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:09,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:09,012 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:09,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:09,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:01:09,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:01:09,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:01:09,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:01:09,013 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:09,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:01:09,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:09,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:09,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:01:09,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:09,014 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:09,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:09,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:01:09,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:09,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:09,040 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:01:09,041 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:01:09,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:01:09,041 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:09,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:01:09,041 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-10-28 00:01:09,041 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-28 00:01:09,041 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:01:09,041 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-10-28 00:01:09,041 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-10-28 00:01:09,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:01:09,041 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-10-28 00:01:09,041 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-10-28 00:01:09,041 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-28 00:01:09,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:01:09,041 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:09,041 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:09,041 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:09,041 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:09,141 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-28 00:01:09,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-28 00:01:09,144 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:01:09,144 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:01:09,145 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. with WAL disabled. Data may be lost in the event of a crash. 2023-10-28 00:01:09,148 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-28 00:01:09,151 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-10-28 00:01:09,151 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-10-28 00:01:09,151 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-10-28 00:01:09,151 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:01:09,151 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-10-28 00:01:09,151 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-10-28 00:01:09,152 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:09,152 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-10-28 00:01:09,162 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:01:09,162 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:09,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:01:09,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:01:09,163 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:09,163 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-28 00:01:09,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-10-28 00:01:09,163 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:09,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-28 00:01:09,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-10-28 00:01:09,164 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-10-28 00:01:09,164 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-10-28 00:01:09,164 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-10-28 00:01:09,164 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-10-28 00:01:09,164 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:01:09,164 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-10-28 00:01:09,164 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. 2023-10-28 00:01:09,164 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. 2023-10-28 00:01:09,164 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. started... 2023-10-28 00:01:09,164 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. 2023-10-28 00:01:09,164 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. started... 2023-10-28 00:01:09,164 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-10-28 00:01:09,164 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-28 00:01:09,164 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. started... 2023-10-28 00:01:09,164 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 1b31b523f7e2913dd7d63aebefeb7c9c 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-10-28 00:01:09,164 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing fdbee376513d41f5e7553c6988d70428 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-10-28 00:01:09,165 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 136ee2e7ad6094dbdbe7b527030e07b6 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-10-28 00:01:09,609 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202310288c2a2b4a8f384f9eb71f8a0410cb7e56 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202310288c2a2b4a8f384f9eb71f8a0410cb7e56 2023-10-28 00:01:09,609 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231028c116bae668004a1b96e26d6a4e348c41 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231028c116bae668004a1b96e26d6a4e348c41 2023-10-28 00:01:09,609 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202310286bdd4f1e13314a11b99d957c0accbe49 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202310286bdd4f1e13314a11b99d957c0accbe49 2023-10-28 00:01:10,023 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/.tmp/family1/bd22580033d04fbdb19fd2699c4f5422 2023-10-28 00:01:10,027 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/.tmp/family1/52e7c00daf074459b48e18baba2f680e 2023-10-28 00:01:10,027 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/.tmp/family1/3acd4916f9ee491abb96a9921e596f7c 2023-10-28 00:01:10,470 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023102832792ec7d8fd4a8eb39fe525474b62e0 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023102832792ec7d8fd4a8eb39fe525474b62e0 2023-10-28 00:01:10,473 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231028b8744ab573d14e658ed87fc4ef5d834b to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20231028b8744ab573d14e658ed87fc4ef5d834b 2023-10-28 00:01:10,473 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202310286a31ef7dc01a40f98a302d1a0148db94 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202310286a31ef7dc01a40f98a302d1a0148db94 2023-10-28 00:01:10,476 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/.tmp/family2/1905ad543f8348b99890499ce441899a 2023-10-28 00:01:10,479 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/.tmp/family2/70b57da054a449c2836e63f18bbc1c25 2023-10-28 00:01:10,480 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/.tmp/family2/19b7bfac05b84098b385d4cc2e6e39e3 2023-10-28 00:01:10,481 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/.tmp/family1/bd22580033d04fbdb19fd2699c4f5422 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/bd22580033d04fbdb19fd2699c4f5422 2023-10-28 00:01:10,484 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/.tmp/family1/52e7c00daf074459b48e18baba2f680e as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/52e7c00daf074459b48e18baba2f680e 2023-10-28 00:01:10,487 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/.tmp/family1/3acd4916f9ee491abb96a9921e596f7c as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/3acd4916f9ee491abb96a9921e596f7c 2023-10-28 00:01:10,510 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/bd22580033d04fbdb19fd2699c4f5422, entries=4, sequenceid=17, filesize=5.5 K 2023-10-28 00:01:10,511 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/.tmp/family2/1905ad543f8348b99890499ce441899a as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/1905ad543f8348b99890499ce441899a 2023-10-28 00:01:10,514 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/52e7c00daf074459b48e18baba2f680e, entries=4, sequenceid=17, filesize=5.5 K 2023-10-28 00:01:10,515 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/.tmp/family2/70b57da054a449c2836e63f18bbc1c25 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/70b57da054a449c2836e63f18bbc1c25 2023-10-28 00:01:10,516 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/1905ad543f8348b99890499ce441899a, entries=2, sequenceid=17, filesize=5.1 K 2023-10-28 00:01:10,517 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/3acd4916f9ee491abb96a9921e596f7c, entries=4, sequenceid=17, filesize=5.5 K 2023-10-28 00:01:10,517 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for fdbee376513d41f5e7553c6988d70428 in 1353ms, sequenceid=17, compaction requested=true 2023-10-28 00:01:10,517 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for fdbee376513d41f5e7553c6988d70428: 2023-10-28 00:01:10,517 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. 2023-10-28 00:01:10,517 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-10-28 00:01:10,518 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/.tmp/family2/19b7bfac05b84098b385d4cc2e6e39e3 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/19b7bfac05b84098b385d4cc2e6e39e3 2023-10-28 00:01:10,521 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/70b57da054a449c2836e63f18bbc1c25, entries=2, sequenceid=17, filesize=5.1 K 2023-10-28 00:01:10,522 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 136ee2e7ad6094dbdbe7b527030e07b6 in 1357ms, sequenceid=17, compaction requested=true 2023-10-28 00:01:10,522 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 136ee2e7ad6094dbdbe7b527030e07b6: 2023-10-28 00:01:10,522 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. 2023-10-28 00:01:10,522 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-10-28 00:01:10,524 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/19b7bfac05b84098b385d4cc2e6e39e3, entries=2, sequenceid=17, filesize=5.1 K 2023-10-28 00:01:10,525 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 1b31b523f7e2913dd7d63aebefeb7c9c in 1361ms, sequenceid=17, compaction requested=true 2023-10-28 00:01:10,525 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 1b31b523f7e2913dd7d63aebefeb7c9c: 2023-10-28 00:01:10,525 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. 2023-10-28 00:01:10,525 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-10-28 00:01:10,525 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-10-28 00:01:10,525 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:10,525 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-10-28 00:01:10,525 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-10-28 00:01:10,560 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:10,560 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:10,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:10,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:01:10,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:01:10,561 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:10,561 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-28 00:01:10,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:01:10,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:01:10,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:10,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:10,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:01:10,563 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-10-28 00:01:10,563 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@25a73d1c[Count = 0] remaining members to acquire global barrier 2023-10-28 00:01:10,563 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-10-28 00:01:10,563 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:10,571 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:10,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:10,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:10,571 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:10,571 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-28 00:01:10,571 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-10-28 00:01:10,571 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-10-28 00:01:10,571 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-28 00:01:10,579 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-10-28 00:01:10,579 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:10,579 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:01:10,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:10,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:01:10,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:01:10,579 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-10-28 00:01:10,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:01:10,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:01:10,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:10,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:10,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:01:10,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:10,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:10,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-28 00:01:10,581 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-10-28 00:01:10,581 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-10-28 00:01:10,582 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-28 00:01:10,582 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-28 00:01:10,582 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-10-28 00:01:10,582 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-28 00:01:10,587 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:10,587 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:10,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:10,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:10,587 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:01:10,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:10,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:01:10,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:01:10,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:01:10,588 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:10,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:01:10,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:01:10,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:10,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:10,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:01:10,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:10,588 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:10,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:10,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:01:10,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:10,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:10,596 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:01:10,596 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:10,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:01:10,596 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:01:10,596 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-10-28 00:01:10,596 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:01:10,596 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-10-28 00:01:10,596 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-28 00:01:10,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:01:10,596 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-10-28 00:01:10,596 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-10-28 00:01:10,596 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:10,596 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:10,596 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-10-28 00:01:10,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:01:10,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:01:10,596 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-28 00:01:10,596 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:10,596 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:10,696 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-28 00:01:10,697 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-28 00:01:10,708 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-10-28 00:01:10,714 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-10-28 00:01:10,721 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-10-28 00:01:10,722 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-10-28 00:01:10,794 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(936): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-10-28 00:01:10,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-10-28 00:01:10,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-10-28 00:01:10,799 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-10-28 00:01:10,799 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:01:10,799 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-10-28 00:01:10,799 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-10-28 00:01:10,799 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:10,799 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-10-28 00:01:10,804 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:01:10,804 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:10,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:01:10,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:01:10,804 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:10,804 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-10-28 00:01:10,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-10-28 00:01:10,805 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:10,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-10-28 00:01:10,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-10-28 00:01:10,805 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-10-28 00:01:10,805 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-10-28 00:01:10,805 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-10-28 00:01:10,805 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-10-28 00:01:10,805 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-10-28 00:01:10,805 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-10-28 00:01:10,806 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. 2023-10-28 00:01:10,806 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. started... 2023-10-28 00:01:10,806 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-10-28 00:01:10,806 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-10-28 00:01:10,806 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. 2023-10-28 00:01:10,806 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 136ee2e7ad6094dbdbe7b527030e07b6 2/2 column families, dataSize=138 B heapSize=944 B 2023-10-28 00:01:10,806 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. 2023-10-28 00:01:10,806 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. started... 2023-10-28 00:01:10,806 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. started... 2023-10-28 00:01:10,806 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 1b31b523f7e2913dd7d63aebefeb7c9c 2/2 column families, dataSize=138 B heapSize=944 B 2023-10-28 00:01:10,806 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing fdbee376513d41f5e7553c6988d70428 2/2 column families, dataSize=138 B heapSize=944 B 2023-10-28 00:01:10,856 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/.tmp/family1/f37c48cf2b6c41baafb94b5ea8741fff 2023-10-28 00:01:10,874 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f37c48cf2b6c41baafb94b5ea8741fff 2023-10-28 00:01:11,657 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/.tmp/family1/051c6a4669bf44099c8c54fc2c4294b4 2023-10-28 00:01:11,658 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/.tmp/family1/5a5d35d3e29e4f63a87ecb9a36a7daef 2023-10-28 00:01:11,686 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 051c6a4669bf44099c8c54fc2c4294b4 2023-10-28 00:01:11,686 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5a5d35d3e29e4f63a87ecb9a36a7daef 2023-10-28 00:01:11,687 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/.tmp/family2/31941118cb7f47209b8ff449e31d213b 2023-10-28 00:01:11,690 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 31941118cb7f47209b8ff449e31d213b 2023-10-28 00:01:11,690 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/.tmp/family1/f37c48cf2b6c41baafb94b5ea8741fff as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/f37c48cf2b6c41baafb94b5ea8741fff 2023-10-28 00:01:11,711 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f37c48cf2b6c41baafb94b5ea8741fff 2023-10-28 00:01:11,711 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/f37c48cf2b6c41baafb94b5ea8741fff, entries=3, sequenceid=23, filesize=5.2 K 2023-10-28 00:01:11,712 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/.tmp/family2/31941118cb7f47209b8ff449e31d213b as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/31941118cb7f47209b8ff449e31d213b 2023-10-28 00:01:11,715 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 31941118cb7f47209b8ff449e31d213b 2023-10-28 00:01:11,715 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/31941118cb7f47209b8ff449e31d213b, entries=1, sequenceid=23, filesize=5.0 K 2023-10-28 00:01:11,716 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for fdbee376513d41f5e7553c6988d70428 in 910ms, sequenceid=23, compaction requested=true 2023-10-28 00:01:11,716 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for fdbee376513d41f5e7553c6988d70428: 2023-10-28 00:01:11,716 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. 2023-10-28 00:01:11,716 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-10-28 00:01:12,506 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/.tmp/family2/ff5c7a86668045a6a498db1cb8ca02b6 2023-10-28 00:01:12,507 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/.tmp/family2/8f64cf146c4a4127abca6435601c3626 2023-10-28 00:01:12,517 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ff5c7a86668045a6a498db1cb8ca02b6 2023-10-28 00:01:12,517 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8f64cf146c4a4127abca6435601c3626 2023-10-28 00:01:12,518 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/.tmp/family1/5a5d35d3e29e4f63a87ecb9a36a7daef as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/5a5d35d3e29e4f63a87ecb9a36a7daef 2023-10-28 00:01:12,518 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/.tmp/family1/051c6a4669bf44099c8c54fc2c4294b4 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/051c6a4669bf44099c8c54fc2c4294b4 2023-10-28 00:01:12,551 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-10-28 00:01:12,551 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x39c29c94 to 127.0.0.1:57369 2023-10-28 00:01:12,551 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-10-28 00:01:12,552 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-10-28 00:01:12,552 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4c6faa71 to 127.0.0.1:57369 2023-10-28 00:01:12,552 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-10-28 00:01:12,553 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 051c6a4669bf44099c8c54fc2c4294b4 2023-10-28 00:01:12,553 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5a5d35d3e29e4f63a87ecb9a36a7daef 2023-10-28 00:01:12,553 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/051c6a4669bf44099c8c54fc2c4294b4, entries=3, sequenceid=23, filesize=5.2 K 2023-10-28 00:01:12,553 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/5a5d35d3e29e4f63a87ecb9a36a7daef, entries=3, sequenceid=23, filesize=5.2 K 2023-10-28 00:01:12,554 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/.tmp/family2/8f64cf146c4a4127abca6435601c3626 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/8f64cf146c4a4127abca6435601c3626 2023-10-28 00:01:12,554 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/.tmp/family2/ff5c7a86668045a6a498db1cb8ca02b6 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/ff5c7a86668045a6a498db1cb8ca02b6 2023-10-28 00:01:12,557 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ff5c7a86668045a6a498db1cb8ca02b6 2023-10-28 00:01:12,557 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8f64cf146c4a4127abca6435601c3626 2023-10-28 00:01:12,557 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/ff5c7a86668045a6a498db1cb8ca02b6, entries=1, sequenceid=23, filesize=5.0 K 2023-10-28 00:01:12,558 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/8f64cf146c4a4127abca6435601c3626, entries=1, sequenceid=23, filesize=5.0 K 2023-10-28 00:01:12,558 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 136ee2e7ad6094dbdbe7b527030e07b6 in 1752ms, sequenceid=23, compaction requested=true 2023-10-28 00:01:12,558 INFO [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 1b31b523f7e2913dd7d63aebefeb7c9c in 1752ms, sequenceid=23, compaction requested=true 2023-10-28 00:01:12,558 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 136ee2e7ad6094dbdbe7b527030e07b6: 2023-10-28 00:01:12,558 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 1b31b523f7e2913dd7d63aebefeb7c9c: 2023-10-28 00:01:12,558 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. 2023-10-28 00:01:12,558 DEBUG [rs(jenkins-hbase16.apache.org,43469,1698451096272)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. 2023-10-28 00:01:12,558 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-10-28 00:01:12,558 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-10-28 00:01:12,558 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-10-28 00:01:12,558 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:12,558 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-10-28 00:01:12,558 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-10-28 00:01:12,604 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:12,604 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:12,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:12,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:01:12,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:01:12,605 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:12,605 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-10-28 00:01:12,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:01:12,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:01:12,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:12,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:12,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:01:12,609 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43469,1698451096272' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-10-28 00:01:12,609 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@70a24a4b[Count = 0] remaining members to acquire global barrier 2023-10-28 00:01:12,609 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-10-28 00:01:12,609 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:12,619 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:12,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:12,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:12,619 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-10-28 00:01:12,619 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-10-28 00:01:12,619 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase16.apache.org,43469,1698451096272' in zk 2023-10-28 00:01:12,620 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:12,620 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-10-28 00:01:12,629 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-10-28 00:01:12,629 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:01:12,630 DEBUG [member: 'jenkins-hbase16.apache.org,43469,1698451096272' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-10-28 00:01:12,629 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:12,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:12,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:01:12,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:01:12,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:01:12,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:01:12,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:12,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:12,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:01:12,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:12,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:12,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase16.apache.org,43469,1698451096272': 2023-10-28 00:01:12,639 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43469,1698451096272' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-10-28 00:01:12,639 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-10-28 00:01:12,639 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-10-28 00:01:12,639 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-10-28 00:01:12,639 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-10-28 00:01:12,639 INFO [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-10-28 00:01:12,646 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:12,646 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:01:12,646 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:12,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:12,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:12,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:12,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-10-28 00:01:12,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-10-28 00:01:12,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:01:12,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:01:12,647 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:12,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-10-28 00:01:12,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:12,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:12,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-10-28 00:01:12,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:12,649 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:12,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:12,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-10-28 00:01:12,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-10-28 00:01:12,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:12,661 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:12,661 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-10-28 00:01:12,661 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-10-28 00:01:12,661 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-10-28 00:01:12,661 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-10-28 00:01:12,661 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-10-28 00:01:12,661 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-10-28 00:01:12,661 DEBUG [(jenkins-hbase16.apache.org,36413,1698451095183)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-10-28 00:01:12,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-10-28 00:01:12,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-10-28 00:01:12,661 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-10-28 00:01:12,661 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-10-28 00:01:12,661 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:12,662 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-10-28 00:01:12,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-10-28 00:01:12,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-10-28 00:01:12,662 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:12,662 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-10-28 00:01:12,663 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-10-28 00:01:12,762 DEBUG [Listener at localhost/40295] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-10-28 00:01:12,764 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36413] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-10-28 00:01:12,765 DEBUG [Listener at localhost/40295] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-10-28 00:01:12,765 DEBUG [Listener at localhost/40295] regionserver.HStore(1912): 136ee2e7ad6094dbdbe7b527030e07b6/family1 is initiating major compaction (all files) 2023-10-28 00:01:12,766 INFO [Listener at localhost/40295] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-10-28 00:01:12,766 INFO [Listener at localhost/40295] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-10-28 00:01:12,766 INFO [Listener at localhost/40295] regionserver.HRegion(2259): Starting compaction of 136ee2e7ad6094dbdbe7b527030e07b6/family1 in testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. 2023-10-28 00:01:12,767 INFO [Listener at localhost/40295] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/c277995f9d4449afba51b623b755bd2c, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/34a67289c63e4e8593c3d6fc7848be8c, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/36b6757862eb47028733e102a535f32c, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/52e7c00daf074459b48e18baba2f680e, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/051c6a4669bf44099c8c54fc2c4294b4] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/.tmp, totalSize=27.1 K 2023-10-28 00:01:12,768 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting c277995f9d4449afba51b623b755bd2c, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698451264387 2023-10-28 00:01:12,769 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 34a67289c63e4e8593c3d6fc7848be8c, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698451266395 2023-10-28 00:01:12,770 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 36b6757862eb47028733e102a535f32c, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698451267535 2023-10-28 00:01:12,772 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 52e7c00daf074459b48e18baba2f680e, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698451269145 2023-10-28 00:01:12,773 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 051c6a4669bf44099c8c54fc2c4294b4, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-10-28 00:01:12,787 INFO [Listener at localhost/40295] throttle.PressureAwareThroughputController(145): 136ee2e7ad6094dbdbe7b527030e07b6#family1#compaction#198 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-10-28 00:01:13,232 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023102809d84d29dd5c42ccb219db3f56676b56_del 2023-10-28 00:01:13,232 INFO [Listener at localhost/40295] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023102809d84d29dd5c42ccb219db3f56676b56_del to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023102809d84d29dd5c42ccb219db3f56676b56_del 2023-10-28 00:01:14,090 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6793bc9327a74e9e9d3c511c3b33189c 2023-10-28 00:01:14,091 DEBUG [Listener at localhost/40295] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/.tmp/family1/6793bc9327a74e9e9d3c511c3b33189c as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/6793bc9327a74e9e9d3c511c3b33189c 2023-10-28 00:01:14,110 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6793bc9327a74e9e9d3c511c3b33189c 2023-10-28 00:01:14,110 INFO [Listener at localhost/40295] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 136ee2e7ad6094dbdbe7b527030e07b6/family1 of 136ee2e7ad6094dbdbe7b527030e07b6 into 6793bc9327a74e9e9d3c511c3b33189c(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-10-28 00:01:14,111 DEBUG [Listener at localhost/40295] regionserver.HRegion(2289): Compaction status journal for 136ee2e7ad6094dbdbe7b527030e07b6: 2023-10-28 00:01:14,111 DEBUG [Listener at localhost/40295] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-10-28 00:01:14,111 DEBUG [Listener at localhost/40295] regionserver.HStore(1912): 136ee2e7ad6094dbdbe7b527030e07b6/family2 is initiating major compaction (all files) 2023-10-28 00:01:14,111 INFO [Listener at localhost/40295] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-10-28 00:01:14,111 INFO [Listener at localhost/40295] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-10-28 00:01:14,111 INFO [Listener at localhost/40295] regionserver.HRegion(2259): Starting compaction of 136ee2e7ad6094dbdbe7b527030e07b6/family2 in testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. 2023-10-28 00:01:14,111 INFO [Listener at localhost/40295] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/9e32ab004ceb44eca3c759b2b6241ec9, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/a1c30a3fbaa545cc9c0a048666af383d, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/ea6a77049ea6401b832c11c52220df30, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/70b57da054a449c2836e63f18bbc1c25, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/8f64cf146c4a4127abca6435601c3626] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/.tmp, totalSize=25.4 K 2023-10-28 00:01:14,112 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 9e32ab004ceb44eca3c759b2b6241ec9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698451264387 2023-10-28 00:01:14,112 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting a1c30a3fbaa545cc9c0a048666af383d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698451266395 2023-10-28 00:01:14,113 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting ea6a77049ea6401b832c11c52220df30, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698451267535 2023-10-28 00:01:14,113 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 70b57da054a449c2836e63f18bbc1c25, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698451269145 2023-10-28 00:01:14,113 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 8f64cf146c4a4127abca6435601c3626, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-10-28 00:01:14,121 INFO [Listener at localhost/40295] throttle.PressureAwareThroughputController(145): 136ee2e7ad6094dbdbe7b527030e07b6#family2#compaction#199 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-10-28 00:01:14,127 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202310286979031891e549f68d93d28c0a6f4b81_del 2023-10-28 00:01:14,127 INFO [Listener at localhost/40295] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202310286979031891e549f68d93d28c0a6f4b81_del to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202310286979031891e549f68d93d28c0a6f4b81_del 2023-10-28 00:01:14,128 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/c277995f9d4449afba51b623b755bd2c, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/34a67289c63e4e8593c3d6fc7848be8c, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/36b6757862eb47028733e102a535f32c, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/52e7c00daf074459b48e18baba2f680e, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/051c6a4669bf44099c8c54fc2c4294b4] to archive 2023-10-28 00:01:14,129 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:01:14,130 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/c277995f9d4449afba51b623b755bd2c to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/c277995f9d4449afba51b623b755bd2c 2023-10-28 00:01:14,131 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/34a67289c63e4e8593c3d6fc7848be8c to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/34a67289c63e4e8593c3d6fc7848be8c 2023-10-28 00:01:14,132 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/36b6757862eb47028733e102a535f32c to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/36b6757862eb47028733e102a535f32c 2023-10-28 00:01:14,133 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/52e7c00daf074459b48e18baba2f680e to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/52e7c00daf074459b48e18baba2f680e 2023-10-28 00:01:14,134 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/051c6a4669bf44099c8c54fc2c4294b4 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/051c6a4669bf44099c8c54fc2c4294b4 2023-10-28 00:01:14,547 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 58861197aae94c599c7358e76f2f4ac3 2023-10-28 00:01:14,549 DEBUG [Listener at localhost/40295] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/.tmp/family2/58861197aae94c599c7358e76f2f4ac3 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/58861197aae94c599c7358e76f2f4ac3 2023-10-28 00:01:14,555 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 58861197aae94c599c7358e76f2f4ac3 2023-10-28 00:01:14,556 INFO [Listener at localhost/40295] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 136ee2e7ad6094dbdbe7b527030e07b6/family2 of 136ee2e7ad6094dbdbe7b527030e07b6 into 58861197aae94c599c7358e76f2f4ac3(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-10-28 00:01:14,556 DEBUG [Listener at localhost/40295] regionserver.HRegion(2289): Compaction status journal for 136ee2e7ad6094dbdbe7b527030e07b6: 2023-10-28 00:01:14,556 DEBUG [Listener at localhost/40295] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-10-28 00:01:14,556 DEBUG [Listener at localhost/40295] regionserver.HStore(1912): fdbee376513d41f5e7553c6988d70428/family1 is initiating major compaction (all files) 2023-10-28 00:01:14,557 INFO [Listener at localhost/40295] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-10-28 00:01:14,557 INFO [Listener at localhost/40295] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-10-28 00:01:14,557 INFO [Listener at localhost/40295] regionserver.HRegion(2259): Starting compaction of fdbee376513d41f5e7553c6988d70428/family1 in testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. 2023-10-28 00:01:14,557 INFO [Listener at localhost/40295] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/4b0eb47973cf4ec198a45ca8917cf70b, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/720a86219f5d474185e2c4182d8e747f, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/ba57df9023b74a44ab9211ff0e235ad3, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/bd22580033d04fbdb19fd2699c4f5422, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/f37c48cf2b6c41baafb94b5ea8741fff] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/.tmp, totalSize=27.1 K 2023-10-28 00:01:14,558 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 4b0eb47973cf4ec198a45ca8917cf70b, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698451264387 2023-10-28 00:01:14,558 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 720a86219f5d474185e2c4182d8e747f, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698451266394 2023-10-28 00:01:14,558 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting ba57df9023b74a44ab9211ff0e235ad3, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698451267534 2023-10-28 00:01:14,559 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting bd22580033d04fbdb19fd2699c4f5422, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698451269144 2023-10-28 00:01:14,559 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting f37c48cf2b6c41baafb94b5ea8741fff, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-10-28 00:01:14,566 INFO [Listener at localhost/40295] throttle.PressureAwareThroughputController(145): fdbee376513d41f5e7553c6988d70428#family1#compaction#200 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-10-28 00:01:14,634 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/9e32ab004ceb44eca3c759b2b6241ec9, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/a1c30a3fbaa545cc9c0a048666af383d, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/ea6a77049ea6401b832c11c52220df30, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/70b57da054a449c2836e63f18bbc1c25, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/8f64cf146c4a4127abca6435601c3626] to archive 2023-10-28 00:01:14,635 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:01:14,636 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/9e32ab004ceb44eca3c759b2b6241ec9 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/9e32ab004ceb44eca3c759b2b6241ec9 2023-10-28 00:01:14,638 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/a1c30a3fbaa545cc9c0a048666af383d to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/a1c30a3fbaa545cc9c0a048666af383d 2023-10-28 00:01:14,639 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/ea6a77049ea6401b832c11c52220df30 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/ea6a77049ea6401b832c11c52220df30 2023-10-28 00:01:14,640 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/70b57da054a449c2836e63f18bbc1c25 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/70b57da054a449c2836e63f18bbc1c25 2023-10-28 00:01:14,641 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/8f64cf146c4a4127abca6435601c3626 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family2/8f64cf146c4a4127abca6435601c3626 2023-10-28 00:01:15,009 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20231028a290a73cf05549349c30c509459f428c_del 2023-10-28 00:01:15,009 INFO [Listener at localhost/40295] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231028a290a73cf05549349c30c509459f428c_del to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231028a290a73cf05549349c30c509459f428c_del 2023-10-28 00:01:15,872 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c03f2885ada541b1b4e46b62401f25ac 2023-10-28 00:01:15,872 DEBUG [Listener at localhost/40295] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/.tmp/family1/c03f2885ada541b1b4e46b62401f25ac as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/c03f2885ada541b1b4e46b62401f25ac 2023-10-28 00:01:15,890 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c03f2885ada541b1b4e46b62401f25ac 2023-10-28 00:01:15,890 INFO [Listener at localhost/40295] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in fdbee376513d41f5e7553c6988d70428/family1 of fdbee376513d41f5e7553c6988d70428 into c03f2885ada541b1b4e46b62401f25ac(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-10-28 00:01:15,891 DEBUG [Listener at localhost/40295] regionserver.HRegion(2289): Compaction status journal for fdbee376513d41f5e7553c6988d70428: 2023-10-28 00:01:15,891 DEBUG [Listener at localhost/40295] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-10-28 00:01:15,891 DEBUG [Listener at localhost/40295] regionserver.HStore(1912): fdbee376513d41f5e7553c6988d70428/family2 is initiating major compaction (all files) 2023-10-28 00:01:15,891 INFO [Listener at localhost/40295] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-10-28 00:01:15,891 INFO [Listener at localhost/40295] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-10-28 00:01:15,892 INFO [Listener at localhost/40295] regionserver.HRegion(2259): Starting compaction of fdbee376513d41f5e7553c6988d70428/family2 in testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. 2023-10-28 00:01:15,892 INFO [Listener at localhost/40295] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/30fbf6fe0f364762b0c7d67d64a97307, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/949be6eeb7e94f4aa18d82b789a298b8, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/2c2508a444b146abaec3416ac7c8b646, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/1905ad543f8348b99890499ce441899a, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/31941118cb7f47209b8ff449e31d213b] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/.tmp, totalSize=25.4 K 2023-10-28 00:01:15,893 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 30fbf6fe0f364762b0c7d67d64a97307, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698451264387 2023-10-28 00:01:15,893 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 949be6eeb7e94f4aa18d82b789a298b8, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698451266394 2023-10-28 00:01:15,893 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 2c2508a444b146abaec3416ac7c8b646, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698451267534 2023-10-28 00:01:15,894 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 1905ad543f8348b99890499ce441899a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698451269144 2023-10-28 00:01:15,894 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 31941118cb7f47209b8ff449e31d213b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-10-28 00:01:15,901 INFO [Listener at localhost/40295] throttle.PressureAwareThroughputController(145): fdbee376513d41f5e7553c6988d70428#family2#compaction#201 average throughput is 1.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-10-28 00:01:15,928 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/4b0eb47973cf4ec198a45ca8917cf70b, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/720a86219f5d474185e2c4182d8e747f, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/ba57df9023b74a44ab9211ff0e235ad3, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/bd22580033d04fbdb19fd2699c4f5422, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/f37c48cf2b6c41baafb94b5ea8741fff] to archive 2023-10-28 00:01:15,929 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:01:15,931 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/4b0eb47973cf4ec198a45ca8917cf70b to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/4b0eb47973cf4ec198a45ca8917cf70b 2023-10-28 00:01:15,934 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/720a86219f5d474185e2c4182d8e747f to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/720a86219f5d474185e2c4182d8e747f 2023-10-28 00:01:15,936 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/ba57df9023b74a44ab9211ff0e235ad3 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/ba57df9023b74a44ab9211ff0e235ad3 2023-10-28 00:01:15,938 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/bd22580033d04fbdb19fd2699c4f5422 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/bd22580033d04fbdb19fd2699c4f5422 2023-10-28 00:01:15,940 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/f37c48cf2b6c41baafb94b5ea8741fff to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/f37c48cf2b6c41baafb94b5ea8741fff 2023-10-28 00:01:16,318 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20231028b486bc1953b14fe5bb7a2791568d206c_del 2023-10-28 00:01:16,318 INFO [Listener at localhost/40295] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231028b486bc1953b14fe5bb7a2791568d206c_del to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20231028b486bc1953b14fe5bb7a2791568d206c_del 2023-10-28 00:01:17,014 DEBUG [regionserver/jenkins-hbase16:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.15384615384615385, tune throughput to 57.69 MB/second 2023-10-28 00:01:17,140 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c12815f28ad34a72943b4530d4083618 2023-10-28 00:01:17,141 DEBUG [Listener at localhost/40295] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/.tmp/family2/c12815f28ad34a72943b4530d4083618 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/c12815f28ad34a72943b4530d4083618 2023-10-28 00:01:17,148 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c12815f28ad34a72943b4530d4083618 2023-10-28 00:01:17,149 INFO [Listener at localhost/40295] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in fdbee376513d41f5e7553c6988d70428/family2 of fdbee376513d41f5e7553c6988d70428 into c12815f28ad34a72943b4530d4083618(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-10-28 00:01:17,149 DEBUG [Listener at localhost/40295] regionserver.HRegion(2289): Compaction status journal for fdbee376513d41f5e7553c6988d70428: 2023-10-28 00:01:17,149 DEBUG [Listener at localhost/40295] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-10-28 00:01:17,149 DEBUG [Listener at localhost/40295] regionserver.HStore(1912): 1b31b523f7e2913dd7d63aebefeb7c9c/family1 is initiating major compaction (all files) 2023-10-28 00:01:17,150 INFO [Listener at localhost/40295] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-10-28 00:01:17,150 INFO [Listener at localhost/40295] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-10-28 00:01:17,150 INFO [Listener at localhost/40295] regionserver.HRegion(2259): Starting compaction of 1b31b523f7e2913dd7d63aebefeb7c9c/family1 in testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. 2023-10-28 00:01:17,150 INFO [Listener at localhost/40295] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/86adf2c58f924b709e65532688c3462d, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/f630f37ee0ab4f0e862cd74c9318a717, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/7b8246041c6d4d4781b4a79f95bf7b5d, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/3acd4916f9ee491abb96a9921e596f7c, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/5a5d35d3e29e4f63a87ecb9a36a7daef] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/.tmp, totalSize=27.1 K 2023-10-28 00:01:17,150 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 86adf2c58f924b709e65532688c3462d, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698451264386 2023-10-28 00:01:17,151 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting f630f37ee0ab4f0e862cd74c9318a717, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698451266394 2023-10-28 00:01:17,152 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 7b8246041c6d4d4781b4a79f95bf7b5d, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698451267534 2023-10-28 00:01:17,152 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 3acd4916f9ee491abb96a9921e596f7c, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698451269144 2023-10-28 00:01:17,153 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 5a5d35d3e29e4f63a87ecb9a36a7daef, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-10-28 00:01:17,163 INFO [Listener at localhost/40295] throttle.PressureAwareThroughputController(145): 1b31b523f7e2913dd7d63aebefeb7c9c#family1#compaction#202 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-10-28 00:01:17,229 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/30fbf6fe0f364762b0c7d67d64a97307, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/949be6eeb7e94f4aa18d82b789a298b8, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/2c2508a444b146abaec3416ac7c8b646, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/1905ad543f8348b99890499ce441899a, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/31941118cb7f47209b8ff449e31d213b] to archive 2023-10-28 00:01:17,230 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:01:17,233 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/30fbf6fe0f364762b0c7d67d64a97307 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/30fbf6fe0f364762b0c7d67d64a97307 2023-10-28 00:01:17,235 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/949be6eeb7e94f4aa18d82b789a298b8 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/949be6eeb7e94f4aa18d82b789a298b8 2023-10-28 00:01:17,238 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/2c2508a444b146abaec3416ac7c8b646 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/2c2508a444b146abaec3416ac7c8b646 2023-10-28 00:01:17,241 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/1905ad543f8348b99890499ce441899a to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/1905ad543f8348b99890499ce441899a 2023-10-28 00:01:17,244 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/31941118cb7f47209b8ff449e31d213b to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family2/31941118cb7f47209b8ff449e31d213b 2023-10-28 00:01:17,608 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20231028e89ffdd188bf4e1ea67a16b44e1a65c4_del 2023-10-28 00:01:17,609 INFO [Listener at localhost/40295] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231028e89ffdd188bf4e1ea67a16b44e1a65c4_del to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231028e89ffdd188bf4e1ea67a16b44e1a65c4_del 2023-10-28 00:01:18,057 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7dc8519db6e24756bef9626654d0532e 2023-10-28 00:01:18,058 DEBUG [Listener at localhost/40295] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/.tmp/family1/7dc8519db6e24756bef9626654d0532e as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/7dc8519db6e24756bef9626654d0532e 2023-10-28 00:01:18,075 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7dc8519db6e24756bef9626654d0532e 2023-10-28 00:01:18,076 INFO [Listener at localhost/40295] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 1b31b523f7e2913dd7d63aebefeb7c9c/family1 of 1b31b523f7e2913dd7d63aebefeb7c9c into 7dc8519db6e24756bef9626654d0532e(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-10-28 00:01:18,076 DEBUG [Listener at localhost/40295] regionserver.HRegion(2289): Compaction status journal for 1b31b523f7e2913dd7d63aebefeb7c9c: 2023-10-28 00:01:18,076 DEBUG [Listener at localhost/40295] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-10-28 00:01:18,076 DEBUG [Listener at localhost/40295] regionserver.HStore(1912): 1b31b523f7e2913dd7d63aebefeb7c9c/family2 is initiating major compaction (all files) 2023-10-28 00:01:18,076 INFO [Listener at localhost/40295] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-10-28 00:01:18,076 INFO [Listener at localhost/40295] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-10-28 00:01:18,076 INFO [Listener at localhost/40295] regionserver.HRegion(2259): Starting compaction of 1b31b523f7e2913dd7d63aebefeb7c9c/family2 in testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. 2023-10-28 00:01:18,076 INFO [Listener at localhost/40295] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/c64f217fda7948d3beefb7b507c68d95, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/cac7b6e36f824fabb794921565bf4d31, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/a45f67a713b544418da5eb04140fc30e, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/19b7bfac05b84098b385d4cc2e6e39e3, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/ff5c7a86668045a6a498db1cb8ca02b6] into tmpdir=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/.tmp, totalSize=25.4 K 2023-10-28 00:01:18,077 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting c64f217fda7948d3beefb7b507c68d95, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698451264386 2023-10-28 00:01:18,077 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting cac7b6e36f824fabb794921565bf4d31, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698451266394 2023-10-28 00:01:18,077 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting a45f67a713b544418da5eb04140fc30e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698451267534 2023-10-28 00:01:18,078 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting 19b7bfac05b84098b385d4cc2e6e39e3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698451269144 2023-10-28 00:01:18,078 DEBUG [Listener at localhost/40295] compactions.Compactor(207): Compacting ff5c7a86668045a6a498db1cb8ca02b6, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-10-28 00:01:18,085 INFO [Listener at localhost/40295] throttle.PressureAwareThroughputController(145): 1b31b523f7e2913dd7d63aebefeb7c9c#family2#compaction#203 average throughput is 1.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-10-28 00:01:18,133 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/86adf2c58f924b709e65532688c3462d, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/f630f37ee0ab4f0e862cd74c9318a717, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/7b8246041c6d4d4781b4a79f95bf7b5d, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/3acd4916f9ee491abb96a9921e596f7c, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/5a5d35d3e29e4f63a87ecb9a36a7daef] to archive 2023-10-28 00:01:18,134 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:01:18,136 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/86adf2c58f924b709e65532688c3462d to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/86adf2c58f924b709e65532688c3462d 2023-10-28 00:01:18,137 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/f630f37ee0ab4f0e862cd74c9318a717 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/f630f37ee0ab4f0e862cd74c9318a717 2023-10-28 00:01:18,138 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/7b8246041c6d4d4781b4a79f95bf7b5d to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/7b8246041c6d4d4781b4a79f95bf7b5d 2023-10-28 00:01:18,139 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/3acd4916f9ee491abb96a9921e596f7c to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/3acd4916f9ee491abb96a9921e596f7c 2023-10-28 00:01:18,140 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/5a5d35d3e29e4f63a87ecb9a36a7daef to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/5a5d35d3e29e4f63a87ecb9a36a7daef 2023-10-28 00:01:18,279 DEBUG [regionserver/jenkins-hbase16:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.15384615384615385, tune throughput to 57.69 MB/second 2023-10-28 00:01:18,492 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202310286acfd2e8e49b4a65ab3532930a0138f2_del 2023-10-28 00:01:18,492 INFO [Listener at localhost/40295] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202310286acfd2e8e49b4a65ab3532930a0138f2_del to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202310286acfd2e8e49b4a65ab3532930a0138f2_del 2023-10-28 00:01:19,315 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a19cdfbaf021496f9bcd0dc3c9893f0a 2023-10-28 00:01:19,317 DEBUG [Listener at localhost/40295] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/.tmp/family2/a19cdfbaf021496f9bcd0dc3c9893f0a as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/a19cdfbaf021496f9bcd0dc3c9893f0a 2023-10-28 00:01:19,324 INFO [Listener at localhost/40295] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a19cdfbaf021496f9bcd0dc3c9893f0a 2023-10-28 00:01:19,325 INFO [Listener at localhost/40295] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 1b31b523f7e2913dd7d63aebefeb7c9c/family2 of 1b31b523f7e2913dd7d63aebefeb7c9c into a19cdfbaf021496f9bcd0dc3c9893f0a(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-10-28 00:01:19,325 DEBUG [Listener at localhost/40295] regionserver.HRegion(2289): Compaction status journal for 1b31b523f7e2913dd7d63aebefeb7c9c: 2023-10-28 00:01:19,330 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-10-28 00:01:19,332 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/c64f217fda7948d3beefb7b507c68d95, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/cac7b6e36f824fabb794921565bf4d31, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/a45f67a713b544418da5eb04140fc30e, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/19b7bfac05b84098b385d4cc2e6e39e3, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/ff5c7a86668045a6a498db1cb8ca02b6] to archive 2023-10-28 00:01:19,333 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:01:19,334 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-10-28 00:01:19,335 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/c64f217fda7948d3beefb7b507c68d95 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/c64f217fda7948d3beefb7b507c68d95 2023-10-28 00:01:19,337 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/cac7b6e36f824fabb794921565bf4d31 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/cac7b6e36f824fabb794921565bf4d31 2023-10-28 00:01:19,338 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/a45f67a713b544418da5eb04140fc30e to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/a45f67a713b544418da5eb04140fc30e 2023-10-28 00:01:19,340 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/19b7bfac05b84098b385d4cc2e6e39e3 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/19b7bfac05b84098b385d4cc2e6e39e3 2023-10-28 00:01:19,340 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-10-28 00:01:19,341 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-10-28 00:01:19,341 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/ff5c7a86668045a6a498db1cb8ca02b6 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family2/ff5c7a86668045a6a498db1cb8ca02b6 2023-10-28 00:01:19,346 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36413] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x97\x0E\xBE`\xA8\xAF\xBF1\x9Ee\xEB\xFFs6\xFD\xB2"\x10\x97h\x83E\xCA\x8C\xC8Q\xBDn\xEAi\xE9\xE2\xE7\xA6*\x10\xE3\x1A\x91\xE8\x0Bx\xF2\x90\xF6\xCC\x84\xF9\x0D\x09\xCD\xFA2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-10-28 00:01:19,367 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] procedure2.ProcedureExecutor(1028): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-10-28 00:01:19,367 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-10-28 00:01:19,368 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-28 00:01:19,368 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698451879368 2023-10-28 00:01:19,368 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-10-28 00:01:19,640 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-10-28 00:01:19,662 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20231028e89ffdd188bf4e1ea67a16b44e1a65c4_del 2023-10-28 00:01:19,681 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20231028a290a73cf05549349c30c509459f428c_del 2023-10-28 00:01:19,700 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023102809d84d29dd5c42ccb219db3f56676b56_del 2023-10-28 00:01:19,700 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-10-28 00:01:19,701 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] zookeeper.ReadOnlyZKClient(139): Connect 0x41cf0dbd to 127.0.0.1:57369 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-10-28 00:01:19,713 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4b2d639b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-28 00:01:19,713 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20231028 for table='testMajorCompactionFromAdmin' and column='family1' 2023-10-28 00:01:20,658 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231028f22734b2c2f3405e826acb5d79662619 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231028f22734b2c2f3405e826acb5d79662619 2023-10-28 00:01:20,659 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231028 table='testMajorCompactionFromAdmin' column='family1' 2023-10-28 00:01:20,669 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-10-28 00:01:20,671 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:46900, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-10-28 00:01:20,678 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-10-28 00:01:20,680 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:34224, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-10-28 00:01:20,708 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20231028/family1/38a1f0880d3d41e7b49be6879afb312c first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-10-28 00:01:20,709 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20231028/family1/38a1f0880d3d41e7b49be6879afb312c}] 2023-10-28 00:01:20,713 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20231028/family1/38a1f0880d3d41e7b49be6879afb312c for inclusion in 1b31b523f7e2913dd7d63aebefeb7c9c/family1 2023-10-28 00:01:20,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-10-28 00:01:20,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(777): Region bounds: first= last=1 2023-10-28 00:01:20,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for 1b31b523f7e2913dd7d63aebefeb7c9c: 2023-10-28 00:01:20,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20231028/family1/38a1f0880d3d41e7b49be6879afb312c to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionFromAdmin__npaonr0j5fo0s3p2r7uvbgrk6kjlpgbaidr6e20p26jbcd2lfb4mlkrntmrfo4kt/family1/38a1f0880d3d41e7b49be6879afb312c 2023-10-28 00:01:20,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionFromAdmin__npaonr0j5fo0s3p2r7uvbgrk6kjlpgbaidr6e20p26jbcd2lfb4mlkrntmrfo4kt/family1/38a1f0880d3d41e7b49be6879afb312c as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/6fee2642ceac4f0086b6133b2ad40b7b_SeqId_28_ 2023-10-28 00:01:20,734 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionFromAdmin__npaonr0j5fo0s3p2r7uvbgrk6kjlpgbaidr6e20p26jbcd2lfb4mlkrntmrfo4kt/family1/38a1f0880d3d41e7b49be6879afb312c into 1b31b523f7e2913dd7d63aebefeb7c9c/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/6fee2642ceac4f0086b6133b2ad40b7b_SeqId_28_ - updating store file list. 2023-10-28 00:01:20,752 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/6fee2642ceac4f0086b6133b2ad40b7b_SeqId_28_ into 1b31b523f7e2913dd7d63aebefeb7c9c/family1 2023-10-28 00:01:20,752 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionFromAdmin__npaonr0j5fo0s3p2r7uvbgrk6kjlpgbaidr6e20p26jbcd2lfb4mlkrntmrfo4kt/family1/38a1f0880d3d41e7b49be6879afb312c into 1b31b523f7e2913dd7d63aebefeb7c9c/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/family1/6fee2642ceac4f0086b6133b2ad40b7b_SeqId_28_) 2023-10-28 00:01:20,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionFromAdmin__npaonr0j5fo0s3p2r7uvbgrk6kjlpgbaidr6e20p26jbcd2lfb4mlkrntmrfo4kt/family1/38a1f0880d3d41e7b49be6879afb312c 2023-10-28 00:01:20,754 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231028 table='testMajorCompactionFromAdmin' column='family1' 2023-10-28 00:01:20,755 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:01:20,756 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023102843811a84fa3b4eaba7538fb89727895a to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023102843811a84fa3b4eaba7538fb89727895a 2023-10-28 00:01:20,757 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202310286d07ffc33ed04d47b18795f47f483eb4 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202310286d07ffc33ed04d47b18795f47f483eb4 2023-10-28 00:01:20,758 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231028c116bae668004a1b96e26d6a4e348c41 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231028c116bae668004a1b96e26d6a4e348c41 2023-10-28 00:01:20,759 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231028dfa1b2d5025843be858ca1b2716f2bd4 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231028dfa1b2d5025843be858ca1b2716f2bd4 2023-10-28 00:01:20,759 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-10-28 00:01:20,759 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20231028' 2023-10-28 00:01:20,760 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20231028 for table='testMajorCompactionFromAdmin' and column='family1' 2023-10-28 00:01:20,760 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20231028 for table='testMajorCompactionFromAdmin' and column='family1' 2023-10-28 00:01:21,706 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231028b346c84d92df44109c40ab3dc2c88631 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231028b346c84d92df44109c40ab3dc2c88631 2023-10-28 00:01:21,707 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231028 table='testMajorCompactionFromAdmin' column='family1' 2023-10-28 00:01:21,742 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20231028/family1/df4f9863912f4906ae8d632a82f8d505 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-10-28 00:01:21,742 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20231028/family1/df4f9863912f4906ae8d632a82f8d505}] 2023-10-28 00:01:21,745 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20231028/family1/df4f9863912f4906ae8d632a82f8d505 for inclusion in fdbee376513d41f5e7553c6988d70428/family1 2023-10-28 00:01:21,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-10-28 00:01:21,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(777): Region bounds: first=1 last=2 2023-10-28 00:01:21,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for fdbee376513d41f5e7553c6988d70428: 2023-10-28 00:01:21,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20231028/family1/df4f9863912f4906ae8d632a82f8d505 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionFromAdmin__sgmjfkrntmgj2rirccpgk9csfmbhgajrloivfmlfaepkqh5g2gqelanlb7tnfmou/family1/df4f9863912f4906ae8d632a82f8d505 2023-10-28 00:01:21,765 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionFromAdmin__sgmjfkrntmgj2rirccpgk9csfmbhgajrloivfmlfaepkqh5g2gqelanlb7tnfmou/family1/df4f9863912f4906ae8d632a82f8d505 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/bcaa7da7a2c94e3ab97dd0196c0b572a_SeqId_28_ 2023-10-28 00:01:21,765 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionFromAdmin__sgmjfkrntmgj2rirccpgk9csfmbhgajrloivfmlfaepkqh5g2gqelanlb7tnfmou/family1/df4f9863912f4906ae8d632a82f8d505 into fdbee376513d41f5e7553c6988d70428/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/bcaa7da7a2c94e3ab97dd0196c0b572a_SeqId_28_ - updating store file list. 2023-10-28 00:01:21,785 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/bcaa7da7a2c94e3ab97dd0196c0b572a_SeqId_28_ into fdbee376513d41f5e7553c6988d70428/family1 2023-10-28 00:01:21,785 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionFromAdmin__sgmjfkrntmgj2rirccpgk9csfmbhgajrloivfmlfaepkqh5g2gqelanlb7tnfmou/family1/df4f9863912f4906ae8d632a82f8d505 into fdbee376513d41f5e7553c6988d70428/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/family1/bcaa7da7a2c94e3ab97dd0196c0b572a_SeqId_28_) 2023-10-28 00:01:21,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionFromAdmin__sgmjfkrntmgj2rirccpgk9csfmbhgajrloivfmlfaepkqh5g2gqelanlb7tnfmou/family1/df4f9863912f4906ae8d632a82f8d505 2023-10-28 00:01:21,787 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231028 table='testMajorCompactionFromAdmin' column='family1' 2023-10-28 00:01:21,788 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:01:21,789 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202310283083d25633e44fe5934ae7a6eb2c1841 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202310283083d25633e44fe5934ae7a6eb2c1841 2023-10-28 00:01:21,790 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202310288c2a2b4a8f384f9eb71f8a0410cb7e56 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202310288c2a2b4a8f384f9eb71f8a0410cb7e56 2023-10-28 00:01:21,791 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231028b935b7280f1f40b3943b19b24f37d105 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231028b935b7280f1f40b3943b19b24f37d105 2023-10-28 00:01:21,792 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231028d20a43169ea2437690666276cab2c746 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231028d20a43169ea2437690666276cab2c746 2023-10-28 00:01:21,792 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-10-28 00:01:21,792 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20231028' 2023-10-28 00:01:21,793 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20231028 for table='testMajorCompactionFromAdmin' and column='family1' 2023-10-28 00:01:21,793 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20231028 for table='testMajorCompactionFromAdmin' and column='family1' 2023-10-28 00:01:22,733 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202310282b9f69e21dfe4a819704ef661015762d to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202310282b9f69e21dfe4a819704ef661015762d 2023-10-28 00:01:22,734 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231028 table='testMajorCompactionFromAdmin' column='family1' 2023-10-28 00:01:22,769 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20231028/family1/85e637d27d56492a8a8b3b7b147cb254 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-10-28 00:01:22,770 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6., hostname=jenkins-hbase16.apache.org,43469,1698451096272, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20231028/family1/85e637d27d56492a8a8b3b7b147cb254}] 2023-10-28 00:01:22,772 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(763): Validating hfile at hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20231028/family1/85e637d27d56492a8a8b3b7b147cb254 for inclusion in 136ee2e7ad6094dbdbe7b527030e07b6/family1 2023-10-28 00:01:22,792 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-10-28 00:01:22,792 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(777): Region bounds: first=2 last= 2023-10-28 00:01:22,792 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegion(2446): Flush status journal for 136ee2e7ad6094dbdbe7b527030e07b6: 2023-10-28 00:01:22,793 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20231028/family1/85e637d27d56492a8a8b3b7b147cb254 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionFromAdmin__o4numqj8j209jp29m3okibonbnv0agc82186j1l1pk7ghp61v34bposf27f7snsq/family1/85e637d27d56492a8a8b3b7b147cb254 2023-10-28 00:01:22,795 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionFromAdmin__o4numqj8j209jp29m3okibonbnv0agc82186j1l1pk7ghp61v34bposf27f7snsq/family1/85e637d27d56492a8a8b3b7b147cb254 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/b0ce44c7f66a4d4f921dfad2ced32828_SeqId_28_ 2023-10-28 00:01:22,795 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(849): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionFromAdmin__o4numqj8j209jp29m3okibonbnv0agc82186j1l1pk7ghp61v34bposf27f7snsq/family1/85e637d27d56492a8a8b3b7b147cb254 into 136ee2e7ad6094dbdbe7b527030e07b6/family1 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/b0ce44c7f66a4d4f921dfad2ced32828_SeqId_28_ - updating store file list. 2023-10-28 00:01:22,816 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(882): Loaded HFile hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/b0ce44c7f66a4d4f921dfad2ced32828_SeqId_28_ into 136ee2e7ad6094dbdbe7b527030e07b6/family1 2023-10-28 00:01:22,816 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.HStore(855): Successfully loaded hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionFromAdmin__o4numqj8j209jp29m3okibonbnv0agc82186j1l1pk7ghp61v34bposf27f7snsq/family1/85e637d27d56492a8a8b3b7b147cb254 into 136ee2e7ad6094dbdbe7b527030e07b6/family1 (new location: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/family1/b0ce44c7f66a4d4f921dfad2ced32828_SeqId_28_) 2023-10-28 00:01:22,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43469] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/staging/jenkins__testMajorCompactionFromAdmin__o4numqj8j209jp29m3okibonbnv0agc82186j1l1pk7ghp61v34bposf27f7snsq/family1/85e637d27d56492a8a8b3b7b147cb254 2023-10-28 00:01:22,818 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231028 table='testMajorCompactionFromAdmin' column='family1' 2023-10-28 00:01:22,818 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:01:22,819 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023102810465475b92342dba5e0cabc6df67358 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023102810465475b92342dba5e0cabc6df67358 2023-10-28 00:01:22,821 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202310286bdd4f1e13314a11b99d957c0accbe49 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202310286bdd4f1e13314a11b99d957c0accbe49 2023-10-28 00:01:22,821 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023102892feee79f01c4fbd9d9510c42ca4c34b to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023102892feee79f01c4fbd9d9510c42ca4c34b 2023-10-28 00:01:22,822 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231028dd2caa04067743be96420810eb0ab746 to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231028dd2caa04067743be96420810eb0ab746 2023-10-28 00:01:22,822 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-10-28 00:01:22,822 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20231028' 2023-10-28 00:01:22,823 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20231028 for table='testMajorCompactionFromAdmin' and column='family1' 2023-10-28 00:01:22,823 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-10-28 00:01:22,823 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-10-28 00:01:22,823 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(367): [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231028e89ffdd188bf4e1ea67a16b44e1a65c4_del] 2023-10-28 00:01:22,823 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:01:22,824 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231028e89ffdd188bf4e1ea67a16b44e1a65c4_del to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231028e89ffdd188bf4e1ea67a16b44e1a65c4_del 2023-10-28 00:01:22,824 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-10-28 00:01:22,824 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(367): [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231028a290a73cf05549349c30c509459f428c_del] 2023-10-28 00:01:22,824 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:01:22,826 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231028a290a73cf05549349c30c509459f428c_del to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231028a290a73cf05549349c30c509459f428c_del 2023-10-28 00:01:22,826 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-10-28 00:01:22,826 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] compactions.PartitionedMobCompactor(367): [hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023102809d84d29dd5c42ccb219db3f56676b56_del] 2023-10-28 00:01:22,826 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] backup.HFileArchiver(360): Archiving compacted files. 2023-10-28 00:01:22,827 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023102809d84d29dd5c42ccb219db3f56676b56_del to hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023102809d84d29dd5c42ccb219db3f56676b56_del 2023-10-28 00:01:22,827 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-10-28 00:01:22,828 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1698451100997] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-10-28 00:01:22,829 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-10-28 00:01:22,830 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 3.4630 sec 2023-10-28 00:01:22,850 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-10-28 00:01:22,854 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-10-28 00:01:22,859 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-10-28 00:01:22,860 DEBUG [Listener at localhost/40295] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-10-28 00:01:22,866 INFO [Listener at localhost/40295] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-10-28 00:01:22,898 INFO [Listener at localhost/40295] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=247 (was 259), OpenFileDescriptor=632 (was 649), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=60 (was 50) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=5888 (was 5897) 2023-10-28 00:01:22,898 DEBUG [Listener at localhost/40295] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2b262d2b to 127.0.0.1:57369 2023-10-28 00:01:22,898 DEBUG [Listener at localhost/40295] ipc.AbstractRpcClient(494): Stopping rpc client 2023-10-28 00:01:22,899 INFO [Listener at localhost/40295] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-10-28 00:01:22,899 INFO [Listener at localhost/40295] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-10-28 00:01:22,899 DEBUG [Listener at localhost/40295] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x22679832 to 127.0.0.1:57369 2023-10-28 00:01:22,899 DEBUG [Listener at localhost/40295] ipc.AbstractRpcClient(494): Stopping rpc client 2023-10-28 00:01:22,899 DEBUG [Listener at localhost/40295] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-10-28 00:01:22,899 DEBUG [Listener at localhost/40295] util.JVMClusterUtil(257): Found active master hash=8953254, stopped=false 2023-10-28 00:01:22,899 INFO [Listener at localhost/40295] master.ServerManager(888): Cluster shutdown requested of master=jenkins-hbase16.apache.org,36413,1698451095183 2023-10-28 00:01:22,943 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-10-28 00:01:22,943 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-10-28 00:01:22,943 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-10-28 00:01:22,943 INFO [Listener at localhost/40295] procedure2.ProcedureExecutor(628): Stopping 2023-10-28 00:01:22,944 DEBUG [Listener at localhost/40295] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1d87fafe to 127.0.0.1:57369 2023-10-28 00:01:22,944 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-10-28 00:01:22,944 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-10-28 00:01:22,944 DEBUG [Listener at localhost/40295] ipc.AbstractRpcClient(494): Stopping rpc client 2023-10-28 00:01:22,945 INFO [Listener at localhost/40295] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase16.apache.org,43469,1698451096272' ***** 2023-10-28 00:01:22,945 INFO [Listener at localhost/40295] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-10-28 00:01:22,945 INFO [RS:0;jenkins-hbase16:43469] regionserver.HeapMemoryManager(220): Stopping 2023-10-28 00:01:22,945 INFO [RS:0;jenkins-hbase16:43469] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-10-28 00:01:22,945 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-10-28 00:01:22,945 INFO [RS:0;jenkins-hbase16:43469] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-10-28 00:01:22,946 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(3305): Received CLOSE for f0ae0de72c21217ffb45facabb5a6cd5 2023-10-28 00:01:22,947 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(3305): Received CLOSE for 4904326d2e05286728bb6d35b919e5bf 2023-10-28 00:01:22,947 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(3305): Received CLOSE for cb61fdd866246978279c7ac1cfc5941f 2023-10-28 00:01:22,947 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing f0ae0de72c21217ffb45facabb5a6cd5, disabling compactions & flushes 2023-10-28 00:01:22,947 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(3305): Received CLOSE for db3340090786ac1f09ea9c45bcb224a0 2023-10-28 00:01:22,947 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. 2023-10-28 00:01:22,947 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(3305): Received CLOSE for 136ee2e7ad6094dbdbe7b527030e07b6 2023-10-28 00:01:22,947 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. 2023-10-28 00:01:22,947 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(3305): Received CLOSE for fdbee376513d41f5e7553c6988d70428 2023-10-28 00:01:22,947 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. after waiting 0 ms 2023-10-28 00:01:22,947 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(3305): Received CLOSE for 1b31b523f7e2913dd7d63aebefeb7c9c 2023-10-28 00:01:22,947 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. 2023-10-28 00:01:22,947 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(3305): Received CLOSE for fec9ad5a4447830583ac360f9df5b068 2023-10-28 00:01:22,948 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(3305): Received CLOSE for 6d789d75232206c2589efabacffe9d96 2023-10-28 00:01:22,948 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(3305): Received CLOSE for d4d6277130ca600bf099d77a935d3b41 2023-10-28 00:01:22,948 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(3305): Received CLOSE for a91badaa245ed444aa843d100a54dfca 2023-10-28 00:01:22,948 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(3305): Received CLOSE for bec4bce8b8bdfe22904d678de5936bd8 2023-10-28 00:01:22,948 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(3305): Received CLOSE for f7ae7da0566a658f8f0ae5d6c67c438c 2023-10-28 00:01:22,948 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(3305): Received CLOSE for f0c80b795ea0c9c5ec8ad26a6060e403 2023-10-28 00:01:22,948 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(3305): Received CLOSE for f8230c1f516a2a5268355abb24416354 2023-10-28 00:01:22,948 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(3305): Received CLOSE for 1591472e0024853e35a0606e0078390d 2023-10-28 00:01:22,948 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(3305): Received CLOSE for 738e5e9e8c019e543eb40425cbae960e 2023-10-28 00:01:22,948 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(3305): Received CLOSE for 5fc23723a82c912e2bb250ea7522b12a 2023-10-28 00:01:22,948 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(3305): Received CLOSE for 795595fcd155993336d014aba73423ea 2023-10-28 00:01:22,948 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(3305): Received CLOSE for e6aa79e78506420304e660bae0e1ad9a 2023-10-28 00:01:22,948 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(1144): stopping server jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:22,948 DEBUG [RS:0;jenkins-hbase16:43469] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x36e87475 to 127.0.0.1:57369 2023-10-28 00:01:22,948 DEBUG [RS:0;jenkins-hbase16:43469] ipc.AbstractRpcClient(494): Stopping rpc client 2023-10-28 00:01:22,949 INFO [RS:0;jenkins-hbase16:43469] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-10-28 00:01:22,949 INFO [RS:0;jenkins-hbase16:43469] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-10-28 00:01:22,949 INFO [RS:0;jenkins-hbase16:43469] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-10-28 00:01:22,949 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-10-28 00:01:22,949 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(1474): Waiting on 21 regions to close 2023-10-28 00:01:22,949 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-10-28 00:01:22,949 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-10-28 00:01:22,949 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-10-28 00:01:22,949 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-10-28 00:01:22,950 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-10-28 00:01:22,950 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=50.62 KB heapSize=78.23 KB 2023-10-28 00:01:22,950 DEBUG [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(1504): Waiting on 136ee2e7ad6094dbdbe7b527030e07b6, 1588230740, 1591472e0024853e35a0606e0078390d, 1b31b523f7e2913dd7d63aebefeb7c9c, 4904326d2e05286728bb6d35b919e5bf, 5fc23723a82c912e2bb250ea7522b12a, 6d789d75232206c2589efabacffe9d96, 738e5e9e8c019e543eb40425cbae960e, 795595fcd155993336d014aba73423ea, a91badaa245ed444aa843d100a54dfca, bec4bce8b8bdfe22904d678de5936bd8, cb61fdd866246978279c7ac1cfc5941f, d4d6277130ca600bf099d77a935d3b41, db3340090786ac1f09ea9c45bcb224a0, e6aa79e78506420304e660bae0e1ad9a, f0ae0de72c21217ffb45facabb5a6cd5, f0c80b795ea0c9c5ec8ad26a6060e403, f7ae7da0566a658f8f0ae5d6c67c438c, f8230c1f516a2a5268355abb24416354, fdbee376513d41f5e7553c6988d70428, fec9ad5a4447830583ac360f9df5b068 2023-10-28 00:01:22,953 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/f0ae0de72c21217ffb45facabb5a6cd5/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-10-28 00:01:22,955 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. 2023-10-28 00:01:22,955 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for f0ae0de72c21217ffb45facabb5a6cd5: 2023-10-28 00:01:22,955 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1698451231205.f0ae0de72c21217ffb45facabb5a6cd5. 2023-10-28 00:01:22,955 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing 4904326d2e05286728bb6d35b919e5bf, disabling compactions & flushes 2023-10-28 00:01:22,955 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:01:22,955 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:01:22,955 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. after waiting 0 ms 2023-10-28 00:01:22,955 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:01:22,958 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/4904326d2e05286728bb6d35b919e5bf/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-10-28 00:01:22,959 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:01:22,960 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for 4904326d2e05286728bb6d35b919e5bf: 2023-10-28 00:01:22,960 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1698451203346.4904326d2e05286728bb6d35b919e5bf. 2023-10-28 00:01:22,960 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing cb61fdd866246978279c7ac1cfc5941f, disabling compactions & flushes 2023-10-28 00:01:22,960 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-28 00:01:22,960 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-28 00:01:22,960 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. after waiting 0 ms 2023-10-28 00:01:22,960 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-28 00:01:22,960 INFO [regionserver/jenkins-hbase16:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-10-28 00:01:22,969 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/cb61fdd866246978279c7ac1cfc5941f/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-10-28 00:01:22,970 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-28 00:01:22,970 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for cb61fdd866246978279c7ac1cfc5941f: 2023-10-28 00:01:22,970 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1698451101129.cb61fdd866246978279c7ac1cfc5941f. 2023-10-28 00:01:22,970 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing db3340090786ac1f09ea9c45bcb224a0, disabling compactions & flushes 2023-10-28 00:01:22,970 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1698451142635.db3340090786ac1f09ea9c45bcb224a0. 2023-10-28 00:01:22,970 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1698451142635.db3340090786ac1f09ea9c45bcb224a0. 2023-10-28 00:01:22,970 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1698451142635.db3340090786ac1f09ea9c45bcb224a0. after waiting 0 ms 2023-10-28 00:01:22,970 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1698451142635.db3340090786ac1f09ea9c45bcb224a0. 2023-10-28 00:01:22,973 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/db3340090786ac1f09ea9c45bcb224a0/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-10-28 00:01:22,974 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1698451142635.db3340090786ac1f09ea9c45bcb224a0. 2023-10-28 00:01:22,974 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for db3340090786ac1f09ea9c45bcb224a0: 2023-10-28 00:01:22,974 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1698451142635.db3340090786ac1f09ea9c45bcb224a0. 2023-10-28 00:01:22,974 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing 136ee2e7ad6094dbdbe7b527030e07b6, disabling compactions & flushes 2023-10-28 00:01:22,974 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. 2023-10-28 00:01:22,974 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. 2023-10-28 00:01:22,974 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. after waiting 0 ms 2023-10-28 00:01:22,974 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. 2023-10-28 00:01:22,977 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/136ee2e7ad6094dbdbe7b527030e07b6/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-10-28 00:01:22,978 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. 2023-10-28 00:01:22,978 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for 136ee2e7ad6094dbdbe7b527030e07b6: 2023-10-28 00:01:22,978 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1698451262240.136ee2e7ad6094dbdbe7b527030e07b6. 2023-10-28 00:01:22,978 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing fdbee376513d41f5e7553c6988d70428, disabling compactions & flushes 2023-10-28 00:01:22,978 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. 2023-10-28 00:01:22,978 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. 2023-10-28 00:01:22,978 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. after waiting 0 ms 2023-10-28 00:01:22,978 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. 2023-10-28 00:01:22,981 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/fdbee376513d41f5e7553c6988d70428/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-10-28 00:01:22,982 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. 2023-10-28 00:01:22,982 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for fdbee376513d41f5e7553c6988d70428: 2023-10-28 00:01:22,982 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1698451262240.fdbee376513d41f5e7553c6988d70428. 2023-10-28 00:01:22,982 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing 1b31b523f7e2913dd7d63aebefeb7c9c, disabling compactions & flushes 2023-10-28 00:01:22,982 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. 2023-10-28 00:01:22,982 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. 2023-10-28 00:01:22,982 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. after waiting 0 ms 2023-10-28 00:01:22,982 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. 2023-10-28 00:01:22,985 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionFromAdmin/1b31b523f7e2913dd7d63aebefeb7c9c/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-10-28 00:01:22,986 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. 2023-10-28 00:01:22,986 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for 1b31b523f7e2913dd7d63aebefeb7c9c: 2023-10-28 00:01:22,986 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1698451262240.1b31b523f7e2913dd7d63aebefeb7c9c. 2023-10-28 00:01:22,986 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing fec9ad5a4447830583ac360f9df5b068, disabling compactions & flushes 2023-10-28 00:01:22,986 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1698451099381.fec9ad5a4447830583ac360f9df5b068. 2023-10-28 00:01:22,986 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1698451099381.fec9ad5a4447830583ac360f9df5b068. 2023-10-28 00:01:22,986 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1698451099381.fec9ad5a4447830583ac360f9df5b068. after waiting 0 ms 2023-10-28 00:01:22,986 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1698451099381.fec9ad5a4447830583ac360f9df5b068. 2023-10-28 00:01:22,986 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2745): Flushing fec9ad5a4447830583ac360f9df5b068 1/1 column families, dataSize=109 B heapSize=592 B 2023-10-28 00:01:23,033 INFO [regionserver/jenkins-hbase16:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-10-28 00:01:23,150 DEBUG [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(1504): Waiting on 1588230740, 1591472e0024853e35a0606e0078390d, 5fc23723a82c912e2bb250ea7522b12a, 6d789d75232206c2589efabacffe9d96, 738e5e9e8c019e543eb40425cbae960e, 795595fcd155993336d014aba73423ea, a91badaa245ed444aa843d100a54dfca, bec4bce8b8bdfe22904d678de5936bd8, d4d6277130ca600bf099d77a935d3b41, e6aa79e78506420304e660bae0e1ad9a, f0c80b795ea0c9c5ec8ad26a6060e403, f7ae7da0566a658f8f0ae5d6c67c438c, f8230c1f516a2a5268355abb24416354, fec9ad5a4447830583ac360f9df5b068 2023-10-28 00:01:23,351 DEBUG [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(1504): Waiting on 1588230740, 1591472e0024853e35a0606e0078390d, 5fc23723a82c912e2bb250ea7522b12a, 6d789d75232206c2589efabacffe9d96, 738e5e9e8c019e543eb40425cbae960e, 795595fcd155993336d014aba73423ea, a91badaa245ed444aa843d100a54dfca, bec4bce8b8bdfe22904d678de5936bd8, d4d6277130ca600bf099d77a935d3b41, e6aa79e78506420304e660bae0e1ad9a, f0c80b795ea0c9c5ec8ad26a6060e403, f7ae7da0566a658f8f0ae5d6c67c438c, f8230c1f516a2a5268355abb24416354, fec9ad5a4447830583ac360f9df5b068 2023-10-28 00:01:23,361 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.34 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/1588230740/.tmp/info/2712b095dfd04b4d8a577b4cf50519d3 2023-10-28 00:01:23,366 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2712b095dfd04b4d8a577b4cf50519d3 2023-10-28 00:01:23,392 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/namespace/fec9ad5a4447830583ac360f9df5b068/.tmp/info/c58f8313314142288f6f0d1c3c8f89e9 2023-10-28 00:01:23,399 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/namespace/fec9ad5a4447830583ac360f9df5b068/.tmp/info/c58f8313314142288f6f0d1c3c8f89e9 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/namespace/fec9ad5a4447830583ac360f9df5b068/info/c58f8313314142288f6f0d1c3c8f89e9 2023-10-28 00:01:23,405 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/namespace/fec9ad5a4447830583ac360f9df5b068/info/c58f8313314142288f6f0d1c3c8f89e9, entries=3, sequenceid=7, filesize=4.8 K 2023-10-28 00:01:23,406 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for fec9ad5a4447830583ac360f9df5b068 in 420ms, sequenceid=7, compaction requested=false 2023-10-28 00:01:23,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/namespace/fec9ad5a4447830583ac360f9df5b068/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-10-28 00:01:23,413 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1698451099381.fec9ad5a4447830583ac360f9df5b068. 2023-10-28 00:01:23,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for fec9ad5a4447830583ac360f9df5b068: 2023-10-28 00:01:23,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1698451099381.fec9ad5a4447830583ac360f9df5b068. 2023-10-28 00:01:23,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing 6d789d75232206c2589efabacffe9d96, disabling compactions & flushes 2023-10-28 00:01:23,413 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:01:23,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:01:23,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. after waiting 0 ms 2023-10-28 00:01:23,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:01:23,418 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/6d789d75232206c2589efabacffe9d96/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-10-28 00:01:23,419 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:01:23,419 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for 6d789d75232206c2589efabacffe9d96: 2023-10-28 00:01:23,419 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1698451203346.6d789d75232206c2589efabacffe9d96. 2023-10-28 00:01:23,420 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing d4d6277130ca600bf099d77a935d3b41, disabling compactions & flushes 2023-10-28 00:01:23,420 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. 2023-10-28 00:01:23,420 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. 2023-10-28 00:01:23,420 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. after waiting 0 ms 2023-10-28 00:01:23,420 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. 2023-10-28 00:01:23,425 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/d4d6277130ca600bf099d77a935d3b41/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-10-28 00:01:23,427 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. 2023-10-28 00:01:23,427 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for d4d6277130ca600bf099d77a935d3b41: 2023-10-28 00:01:23,427 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1698451231205.d4d6277130ca600bf099d77a935d3b41. 2023-10-28 00:01:23,427 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing a91badaa245ed444aa843d100a54dfca, disabling compactions & flushes 2023-10-28 00:01:23,427 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1698451142635.a91badaa245ed444aa843d100a54dfca. 2023-10-28 00:01:23,427 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1698451142635.a91badaa245ed444aa843d100a54dfca. 2023-10-28 00:01:23,427 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1698451142635.a91badaa245ed444aa843d100a54dfca. after waiting 0 ms 2023-10-28 00:01:23,427 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1698451142635.a91badaa245ed444aa843d100a54dfca. 2023-10-28 00:01:23,431 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/a91badaa245ed444aa843d100a54dfca/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-10-28 00:01:23,433 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1698451142635.a91badaa245ed444aa843d100a54dfca. 2023-10-28 00:01:23,433 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for a91badaa245ed444aa843d100a54dfca: 2023-10-28 00:01:23,433 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1698451142635.a91badaa245ed444aa843d100a54dfca. 2023-10-28 00:01:23,433 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing bec4bce8b8bdfe22904d678de5936bd8, disabling compactions & flushes 2023-10-28 00:01:23,433 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-28 00:01:23,433 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-28 00:01:23,433 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. after waiting 0 ms 2023-10-28 00:01:23,433 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-28 00:01:23,438 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bec4bce8b8bdfe22904d678de5936bd8/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-10-28 00:01:23,439 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-28 00:01:23,439 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for bec4bce8b8bdfe22904d678de5936bd8: 2023-10-28 00:01:23,439 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698451178607.bec4bce8b8bdfe22904d678de5936bd8. 2023-10-28 00:01:23,439 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing f7ae7da0566a658f8f0ae5d6c67c438c, disabling compactions & flushes 2023-10-28 00:01:23,439 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-28 00:01:23,439 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-28 00:01:23,440 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. after waiting 0 ms 2023-10-28 00:01:23,440 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-28 00:01:23,443 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithWeeklyPolicy/f7ae7da0566a658f8f0ae5d6c67c438c/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-10-28 00:01:23,445 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-28 00:01:23,445 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for f7ae7da0566a658f8f0ae5d6c67c438c: 2023-10-28 00:01:23,445 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1698451130056.f7ae7da0566a658f8f0ae5d6c67c438c. 2023-10-28 00:01:23,445 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing f0c80b795ea0c9c5ec8ad26a6060e403, disabling compactions & flushes 2023-10-28 00:01:23,445 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403. 2023-10-28 00:01:23,445 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403. 2023-10-28 00:01:23,445 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403. after waiting 0 ms 2023-10-28 00:01:23,445 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403. 2023-10-28 00:01:23,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testScannerOnBulkLoadRefHFiles/f0c80b795ea0c9c5ec8ad26a6060e403/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-10-28 00:01:23,449 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403. 2023-10-28 00:01:23,449 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for f0c80b795ea0c9c5ec8ad26a6060e403: 2023-10-28 00:01:23,449 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1698451142635.f0c80b795ea0c9c5ec8ad26a6060e403. 2023-10-28 00:01:23,449 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing f8230c1f516a2a5268355abb24416354, disabling compactions & flushes 2023-10-28 00:01:23,449 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:01:23,449 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:01:23,449 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. after waiting 0 ms 2023-10-28 00:01:23,449 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:01:23,453 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMinorCompactionWithMonthlyPolicy/f8230c1f516a2a5268355abb24416354/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-10-28 00:01:23,454 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:01:23,454 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for f8230c1f516a2a5268355abb24416354: 2023-10-28 00:01:23,454 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1698451248491.f8230c1f516a2a5268355abb24416354. 2023-10-28 00:01:23,454 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing 1591472e0024853e35a0606e0078390d, disabling compactions & flushes 2023-10-28 00:01:23,454 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-28 00:01:23,454 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-28 00:01:23,454 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. after waiting 0 ms 2023-10-28 00:01:23,454 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-28 00:01:23,458 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithMonthlyPolicy/1591472e0024853e35a0606e0078390d/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-10-28 00:01:23,459 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-28 00:01:23,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for 1591472e0024853e35a0606e0078390d: 2023-10-28 00:01:23,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1698451116700.1591472e0024853e35a0606e0078390d. 2023-10-28 00:01:23,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing 738e5e9e8c019e543eb40425cbae960e, disabling compactions & flushes 2023-10-28 00:01:23,459 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-28 00:01:23,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-28 00:01:23,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. after waiting 0 ms 2023-10-28 00:01:23,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-28 00:01:23,463 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyPolicy/738e5e9e8c019e543eb40425cbae960e/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-10-28 00:01:23,464 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-28 00:01:23,464 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for 738e5e9e8c019e543eb40425cbae960e: 2023-10-28 00:01:23,464 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1698451146494.738e5e9e8c019e543eb40425cbae960e. 2023-10-28 00:01:23,464 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing 5fc23723a82c912e2bb250ea7522b12a, disabling compactions & flushes 2023-10-28 00:01:23,464 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-28 00:01:23,464 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-28 00:01:23,464 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. after waiting 0 ms 2023-10-28 00:01:23,464 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-28 00:01:23,468 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/5fc23723a82c912e2bb250ea7522b12a/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-10-28 00:01:23,469 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-28 00:01:23,469 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for 5fc23723a82c912e2bb250ea7522b12a: 2023-10-28 00:01:23,469 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698451160962.5fc23723a82c912e2bb250ea7522b12a. 2023-10-28 00:01:23,469 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing 795595fcd155993336d014aba73423ea, disabling compactions & flushes 2023-10-28 00:01:23,469 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:01:23,469 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:01:23,469 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. after waiting 0 ms 2023-10-28 00:01:23,469 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:01:23,473 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/default/testCompactionWithHFileLink/795595fcd155993336d014aba73423ea/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-10-28 00:01:23,474 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:01:23,474 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for 795595fcd155993336d014aba73423ea: 2023-10-28 00:01:23,474 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1698451203346.795595fcd155993336d014aba73423ea. 2023-10-28 00:01:23,474 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1604): Closing e6aa79e78506420304e660bae0e1ad9a, disabling compactions & flushes 2023-10-28 00:01:23,474 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. 2023-10-28 00:01:23,474 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. 2023-10-28 00:01:23,474 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. after waiting 0 ms 2023-10-28 00:01:23,474 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. 2023-10-28 00:01:23,478 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/ns/testMinorCompaction/e6aa79e78506420304e660bae0e1ad9a/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-10-28 00:01:23,478 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. 2023-10-28 00:01:23,478 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for e6aa79e78506420304e660bae0e1ad9a: 2023-10-28 00:01:23,478 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1698451231205.e6aa79e78506420304e660bae0e1ad9a. 2023-10-28 00:01:23,551 DEBUG [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-10-28 00:01:23,752 DEBUG [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-10-28 00:01:23,772 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/1588230740/.tmp/rep_barrier/6525b5fb21fc42b78609b41c08609533 2023-10-28 00:01:23,783 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6525b5fb21fc42b78609b41c08609533 2023-10-28 00:01:23,790 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/1588230740/.tmp/table/0b6a08234b314564a0396552acbd9069 2023-10-28 00:01:23,795 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0b6a08234b314564a0396552acbd9069 2023-10-28 00:01:23,796 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/1588230740/.tmp/info/2712b095dfd04b4d8a577b4cf50519d3 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/1588230740/info/2712b095dfd04b4d8a577b4cf50519d3 2023-10-28 00:01:23,800 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2712b095dfd04b4d8a577b4cf50519d3 2023-10-28 00:01:23,801 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/1588230740/info/2712b095dfd04b4d8a577b4cf50519d3, entries=226, sequenceid=130, filesize=37.9 K 2023-10-28 00:01:23,801 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/1588230740/.tmp/rep_barrier/6525b5fb21fc42b78609b41c08609533 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/1588230740/rep_barrier/6525b5fb21fc42b78609b41c08609533 2023-10-28 00:01:23,805 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6525b5fb21fc42b78609b41c08609533 2023-10-28 00:01:23,805 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/1588230740/rep_barrier/6525b5fb21fc42b78609b41c08609533, entries=1, sequenceid=130, filesize=5.0 K 2023-10-28 00:01:23,806 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/1588230740/.tmp/table/0b6a08234b314564a0396552acbd9069 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/1588230740/table/0b6a08234b314564a0396552acbd9069 2023-10-28 00:01:23,809 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0b6a08234b314564a0396552acbd9069 2023-10-28 00:01:23,809 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/1588230740/table/0b6a08234b314564a0396552acbd9069, entries=27, sequenceid=130, filesize=6.6 K 2023-10-28 00:01:23,811 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50.62 KB/51839, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 861ms, sequenceid=130, compaction requested=false 2023-10-28 00:01:23,819 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-10-28 00:01:23,819 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-10-28 00:01:23,819 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-10-28 00:01:23,819 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-10-28 00:01:23,819 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-10-28 00:01:23,933 INFO [regionserver/jenkins-hbase16:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-10-28 00:01:23,933 INFO [regionserver/jenkins-hbase16:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-10-28 00:01:23,952 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(1170): stopping server jenkins-hbase16.apache.org,43469,1698451096272; all regions closed. 2023-10-28 00:01:23,965 DEBUG [RS:0;jenkins-hbase16:43469] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/oldWALs 2023-10-28 00:01:23,965 INFO [RS:0;jenkins-hbase16:43469] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase16.apache.org%2C43469%2C1698451096272.meta:.meta(num 1698451099104) 2023-10-28 00:01:23,971 DEBUG [RS:0;jenkins-hbase16:43469] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/oldWALs 2023-10-28 00:01:23,971 INFO [RS:0;jenkins-hbase16:43469] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase16.apache.org%2C43469%2C1698451096272:(num 1698451098080) 2023-10-28 00:01:23,971 DEBUG [RS:0;jenkins-hbase16:43469] ipc.AbstractRpcClient(494): Stopping rpc client 2023-10-28 00:01:23,971 INFO [RS:0;jenkins-hbase16:43469] regionserver.LeaseManager(133): Closed leases 2023-10-28 00:01:23,971 INFO [RS:0;jenkins-hbase16:43469] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase16:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-10-28 00:01:23,972 INFO [regionserver/jenkins-hbase16:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-10-28 00:01:23,973 INFO [RS:0;jenkins-hbase16:43469] ipc.NettyRpcServer(158): Stopping server on /188.40.62.62:43469 2023-10-28 00:01:24,012 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-10-28 00:01:24,012 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase16.apache.org,43469,1698451096272 2023-10-28 00:01:24,012 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-10-28 00:01:24,013 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase16.apache.org,43469,1698451096272] 2023-10-28 00:01:24,014 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase16.apache.org,43469,1698451096272; numProcessing=1 2023-10-28 00:01:24,029 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase16.apache.org,43469,1698451096272 already deleted, retry=false 2023-10-28 00:01:24,029 INFO [RegionServerTracker-0] master.ServerManager(554): Cluster shutdown set; jenkins-hbase16.apache.org,43469,1698451096272 expired; onlineServers=0 2023-10-28 00:01:24,029 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase16.apache.org,36413,1698451095183' ***** 2023-10-28 00:01:24,029 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-10-28 00:01:24,030 DEBUG [M:0;jenkins-hbase16:36413] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@41b5979d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase16.apache.org/188.40.62.62:0 2023-10-28 00:01:24,030 INFO [M:0;jenkins-hbase16:36413] regionserver.HRegionServer(1144): stopping server jenkins-hbase16.apache.org,36413,1698451095183 2023-10-28 00:01:24,030 INFO [M:0;jenkins-hbase16:36413] regionserver.HRegionServer(1170): stopping server jenkins-hbase16.apache.org,36413,1698451095183; all regions closed. 2023-10-28 00:01:24,030 DEBUG [M:0;jenkins-hbase16:36413] ipc.AbstractRpcClient(494): Stopping rpc client 2023-10-28 00:01:24,031 DEBUG [M:0;jenkins-hbase16:36413] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-10-28 00:01:24,031 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-10-28 00:01:24,031 DEBUG [M:0;jenkins-hbase16:36413] cleaner.HFileCleaner(317): Stopping file delete threads 2023-10-28 00:01:24,031 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.large.0-1698451097774] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.large.0-1698451097774,5,FailOnTimeoutGroup] 2023-10-28 00:01:24,031 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.small.0-1698451097775] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.small.0-1698451097775,5,FailOnTimeoutGroup] 2023-10-28 00:01:24,032 INFO [M:0;jenkins-hbase16:36413] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-10-28 00:01:24,032 INFO [M:0;jenkins-hbase16:36413] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-10-28 00:01:24,032 INFO [M:0;jenkins-hbase16:36413] hbase.ChoreService(369): Chore service for: master/jenkins-hbase16:0 had [] on shutdown 2023-10-28 00:01:24,033 DEBUG [M:0;jenkins-hbase16:36413] master.HMaster(1512): Stopping service threads 2023-10-28 00:01:24,033 INFO [M:0;jenkins-hbase16:36413] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-10-28 00:01:24,034 ERROR [M:0;jenkins-hbase16:36413] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-20,5,PEWorkerGroup] Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-10-28 00:01:24,035 INFO [M:0;jenkins-hbase16:36413] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-10-28 00:01:24,035 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-10-28 00:01:24,037 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-10-28 00:01:24,037 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-10-28 00:01:24,037 DEBUG [M:0;jenkins-hbase16:36413] zookeeper.ZKUtil(398): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-10-28 00:01:24,038 WARN [M:0;jenkins-hbase16:36413] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-10-28 00:01:24,038 INFO [M:0;jenkins-hbase16:36413] assignment.AssignmentManager(315): Stopping assignment manager 2023-10-28 00:01:24,038 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-10-28 00:01:24,038 INFO [M:0;jenkins-hbase16:36413] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-10-28 00:01:24,038 DEBUG [M:0;jenkins-hbase16:36413] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-10-28 00:01:24,039 INFO [M:0;jenkins-hbase16:36413] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-10-28 00:01:24,039 DEBUG [M:0;jenkins-hbase16:36413] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-10-28 00:01:24,039 DEBUG [M:0;jenkins-hbase16:36413] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-10-28 00:01:24,039 DEBUG [M:0;jenkins-hbase16:36413] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-10-28 00:01:24,039 INFO [M:0;jenkins-hbase16:36413] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=513.26 KB heapSize=599.43 KB 2023-10-28 00:01:24,121 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-28 00:01:24,121 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): regionserver:43469-0x1018a51ee3f0001, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-28 00:01:24,121 INFO [RS:0;jenkins-hbase16:43469] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase16.apache.org,43469,1698451096272; zookeeper connection closed. 2023-10-28 00:01:24,121 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@1f679add] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@1f679add 2023-10-28 00:01:24,122 INFO [Listener at localhost/40295] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-10-28 00:01:24,450 INFO [M:0;jenkins-hbase16:36413] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=513.26 KB at sequenceid=1011 (bloomFilter=true), to=hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/ff910fef276447b091424371f9912fe9 2023-10-28 00:01:24,488 INFO [M:0;jenkins-hbase16:36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ff910fef276447b091424371f9912fe9 2023-10-28 00:01:24,490 DEBUG [M:0;jenkins-hbase16:36413] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/ff910fef276447b091424371f9912fe9 as hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/ff910fef276447b091424371f9912fe9 2023-10-28 00:01:24,497 INFO [M:0;jenkins-hbase16:36413] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ff910fef276447b091424371f9912fe9 2023-10-28 00:01:24,497 INFO [M:0;jenkins-hbase16:36413] regionserver.HStore(1080): Added hdfs://localhost:44261/user/jenkins/test-data/63e120c6-15be-3de6-7886-9512e84a5234/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/ff910fef276447b091424371f9912fe9, entries=123, sequenceid=1011, filesize=33.3 K 2023-10-28 00:01:24,498 INFO [M:0;jenkins-hbase16:36413] regionserver.HRegion(2948): Finished flush of dataSize ~513.26 KB/525580, heapSize ~599.41 KB/613800, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 459ms, sequenceid=1011, compaction requested=false 2023-10-28 00:01:24,499 INFO [M:0;jenkins-hbase16:36413] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-10-28 00:01:24,499 DEBUG [M:0;jenkins-hbase16:36413] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-10-28 00:01:24,502 INFO [M:0;jenkins-hbase16:36413] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-10-28 00:01:24,502 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-10-28 00:01:24,502 INFO [M:0;jenkins-hbase16:36413] ipc.NettyRpcServer(158): Stopping server on /188.40.62.62:36413 2023-10-28 00:01:24,520 DEBUG [M:0;jenkins-hbase16:36413] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase16.apache.org,36413,1698451095183 already deleted, retry=false 2023-10-28 00:01:24,629 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-28 00:01:24,629 INFO [M:0;jenkins-hbase16:36413] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase16.apache.org,36413,1698451095183; zookeeper connection closed. 2023-10-28 00:01:24,629 DEBUG [Listener at localhost/40295-EventThread] zookeeper.ZKWatcher(600): master:36413-0x1018a51ee3f0000, quorum=127.0.0.1:57369, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-28 00:01:24,633 WARN [Listener at localhost/40295] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-10-28 00:01:24,640 INFO [Listener at localhost/40295] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-10-28 00:01:24,754 WARN [BP-217416042-188.40.62.62-1698451092001 heartbeating to localhost/127.0.0.1:44261] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-10-28 00:01:24,754 WARN [BP-217416042-188.40.62.62-1698451092001 heartbeating to localhost/127.0.0.1:44261] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-217416042-188.40.62.62-1698451092001 (Datanode Uuid 0a9911f5-af3f-4cd3-a291-4cd50df9aa82) service to localhost/127.0.0.1:44261 2023-10-28 00:01:24,757 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/cluster_1ba21e36-f5ca-27f1-0277-ca251e3b747a/dfs/data/data1/current/BP-217416042-188.40.62.62-1698451092001] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-10-28 00:01:24,757 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/fa09aab1-0166-5acf-e9b9-cb0982f2a33c/cluster_1ba21e36-f5ca-27f1-0277-ca251e3b747a/dfs/data/data2/current/BP-217416042-188.40.62.62-1698451092001] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-10-28 00:01:24,786 INFO [Listener at localhost/40295] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-10-28 00:01:24,916 INFO [Listener at localhost/40295] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-10-28 00:01:24,991 INFO [Listener at localhost/40295] hbase.HBaseTestingUtility(1293): Minicluster is down